with , for , and a sequence of nonnegative stepsizes. We can rewrite , for , where is a martingale difference sequence with respect to , i.e., , a.s., for all . We have the following theorem.

Theorem 1 Suppose that the set of minima of of is non empty, and that the stochastic subgradients satisfy

Moreover, assume that , . Then the sequence of iterates following (1) converges almost surely to some minimum .

In various papers and books and books on stochastic approximations, e.g. [1-4], one can find very general theorems on the convergence of algorithms such as (1), including various additional error terms. Still, trying to apply these general theorems to the simple situation considered here, you might find that you need to make additional assumptions, for example that is continuously differentiable, that is a single point, and/or that you know somehow a priori that the sequence is bounded almost surely. The assumptions on and might not hold here (consider for example ), and the extra work required to prove that is bounded is essentially of the same order as proving the theorem from scratch in this case. This is not to say that Theorem 1 cannot be found in the literature. For example, the proof below adapts one found for a somewhat more specific context in [5]. But in my opinion a good book on stochastic approximation algorithms should reference such a basic and widely applicable theorem more clearly. The proof presented here uses the following useful version of the supermartingale convergence theorem, see [6].

Theorem 2 (Supermartingale Convergence Theorem) Let be three sequences of random variables and let be a filtration (i.e., sigma algebras such that ). Suppose that

1. The random variables are nonnegative and -measurable.
2. For each , we have .
3. There holds .

Then, we have , and the sequence converges to a nonnegative random variable , with probability .

Proof: (of the stochastic subgradient convergence theorem) For and , we have by definition of a subgradient

Hence

This inequality is fundamental to study the progress made by many subgradient algorithms. We use it first by taking , for some , to get

Now by the supermartingale convergence theorem, we deduce that almost surely, and the sequence converges almost surely. In particular, is bounded almost surely.

The first conclusion implies immediately that

and it only remains to show that truly converges to one of the minima, rather than just to the set . For this, take a countable set of points in , dense in . For each such point , we have as above that converges. Hence with probability , all sequences for converge. Since the sequence is bounded, it has at least one limit point. This limit point must be unique in view of the convergence of all the sequences . Hence converges.

[1] A. Benveniste, M. Métivier, P. Priouret, Adaptive Algorithms and Stochastic Approximations. Springer, 1990.

[2] H. J. Kushner and G. G. Yin, Stochastic Approximation and Recursive Algorithms and Applications. Springer, 2nd Edition, 2003.

[3] J. C. Spall, Introduction to Stochastic Search and Optimization. Wiley-Interscience, 2003.

[4] V. S. Borkar Stochastic Approximation: A Dynamical Systems Viewpoint. Cambridge University Press, 2008.

[5] D. P. Bertsekas and A. Nedic and A. E. Ozdaglar, Convex Analysis and Optimization. Athena Scientific, 2003.

[6] J. Neveu, Discete Parameter Martingales, North-Holland, 1975.

sudo cabal install glfw -fdynamic

The “dynamic” flag avoids that cabal tries to recompile its version of GLFW.

Update 11/02/10: Aviation week has another interesting article on systems engineering and integration, which mentions the META program.

I struggled quite a bit yesterday to compile the library on my Mac running Snow Leopard though. Here is the final configuration I used, and so far that seems to be working (at least make test succeeded):

./configure FFLAGS="-arch x86_64" ADD_CXXFLAGS="-mmacosx-version-min=10.4" \
ADD_CFLAGS="-mmacosx-version-min=10.4" ADD_FFLAGS="-mmacosx-version-min=10.4" \
LDFLAGS="-flat_namespace" --with-blas='-framework vecLib' \
--with-lapack='-framework vecLib'

While I’m at it, if you want to compile Ipopt, the coin-or interior point optimizer on which Bonmin relies, then the following simpler configuration should do it:

./configure FFLAGS="-arch x86_64" --with-blas='-framework vecLib' \
--with-lapack='-framework vecLib'

ADS-B is an important technological component of NextGen, as it will allow more people, including the pilots, to have better information about the current state of the system. In fact, commentaries often equate ADS-B with NextGen, although according to the FAA, NextGen is just an “umbrella term for the ongoing, wide-ranging transformation of the National Airspace System (NAS)”. The ultimate goal of NextGen is of course not  simply track aircraft better. Rather, it should improve safety, reduce air traffic congestion and hence result is lower fuel consumed and shorter flight times, etc. To control the National Airspace System, it is clearly beneficial to have better sensors such as the ADS-B system. Ultimately however, this will not be sufficient to improve the performance of the system: better large-scale decision and control algorithms that use this information will need to be accepted and applied by the air traffic controllers and airlines.

[1] J.R. Wilson, “NextGen: A Slow Transformation”, Aerospace America Magazine, May 2010.

The conference was fun. For those who wonder, CPS stands for “Cyber-Physical System”. Granted, that might not be the best name one could come up with. I actually thought that “cyber-” had disappeared in the 90s, but apparently it has come back. Pretty much every plenary speaker struggled to offer his definition of what a CPS is, but none was very convincing.

So let me maybe say here what CPS are in my opinion. It turns out that many researchers working in different areas, such as real-time systems, control systems, or sensor networks, face the same kind of problems and work on the same systems, but used to go to different conferences and publish in different journals. The goal of CPS week is to change that and foster collaboration, which I think is a good idea. But you need a new word to be able to bring these people together. Clearly, you cannot suddenly tell researchers in real-time systems that now they are doing control theory, and vice-versa. That’s why you can pretty much put whatever you want in your definition of CPS, depending on your background.

Nevertheless, the main drawback of Matlab is that is it not free, and there has has been a desire to develop more widely accessible alternatives. The first that come to mind are Scilab and Octave. However, despite their qualities these packages have suffered from an insufficient number of users for their usage to become more widespread. For example, Scilab was created in 2003 by INRIA (the French national institute for research in computer science and control) but the overwhelming majority of the Scilab consortium still consists of French research institutes and companies. I believe that one important reason why the situation is unlikely to change is that MATLAB is universally used in the classroom to teach control systems design, in part because it is expected that students entering the industry know this language. Still, this is not a very good explanation: Scilab and Octave are purposely using a syntax very similar to MATLAB.

Anyway, another issue with MATLAB is that it is yet another language to learn, and new features in the language tend to be introduced slowly. For example, object-oriented design has been added only recently. Instead, there is a strong trend currently in using Python for scientific computing, and several scientific communities are enthusiastically developing in Python. Some examples of impressive packages include Numpy and Scipy, Matplotlib for visualization, and Sage, which is a wrapper for a large number of mathematical libraries. One can find an increasing number of libraries for signal processing, machine learning, computer vision, optimization, and so on. With regard to control systems libraries however, there seems to be much less activity. Here are some related links that I know of:

• Python control systems, with only one developper, Rafael G. Martins.
• Richard Murray’s page on a control systems library for Python
• Stephen Boyd and Lieven Vandenberghe are now mostly developing optimization libraries, but a few applications specific to control systems can be found on their pages.
• PyDSTool: not exactly a control systems library, but a simulation, modeling and analysis package for dynamical systems. It is related, but not a port, to the DsTool software package developed some time ago for analyzing dynamical systems. PyDSTool seems to support hybrid systems.
• Simpy: a discrete-event simulation language.
• Pyro: a Python robotics library.

Please let me know of any other Python library related to control systems.

And a funny new UAV, developped by a German team. The applications would be the usual ones, surveillance and emergency communications. But what’s more interesting is that there is absolutely no pilot in the loop here controlling the aircraft remotely. So this leaves us with a lot of opportunities to improve autonomy technology and safe integration into the airspace.