documenting

nantonel · nantonel · commit a3aab8fcfc14 · 2018-03-06T18:18:45.000+01:00
diff --git a/docs/make.jl b/docs/make.jl
@@ -6,11 +6,11 @@ makedocs(
   sitename = "StructuredOptimization",
   authors = "Niccolò Antonello and Lorenzo Stella",
   pages = Any[
-  "Home"            => "index.md",
-  "Guide"           => "tutorial.md",
-  "Expressions"     => "expressions.md",
-  "Functions"       => "functions.md",
-  "Solvers"         => "solvers.md",
+  "Home"                  => "index.md",
+  "Quick Tutorial Guide"  => "tutorial.md",
+  "Expressions"           => "expressions.md",
+  "Functions"             => "functions.md",
+  "Solvers"               => "solvers.md",
   ],
 )
 
diff --git a/docs/src/expressions.md b/docs/src/expressions.md
@@ -1,12 +1,21 @@
 # Expressions
 
+With `StructuredOptimization.jl` you can easily create mathematical expressions. 
+
+Firstly, [Variables](@ref) must be defined: various [Mappings](@ref) can then be applied 
+following the application of [Functions and constraints](@ref) to create the `Term`s  that define the optimization problem. 
+
 ## Variables
 
 ### Creating Variables
 
 ```@docs
 Variable
 ```
+!!! note 
+
+    `StructuredOptimization.jl` supports complex variables. It is possible to create them by specifying the type 
+    `Variable(Complex{Float64}, 10)` or by initializing them with a complex array `Variable(randn(10)+im*randn(10))`.
 
 ### Utilities
 
@@ -28,15 +37,22 @@ eltype
 *
 ```
 
-## Operators
+## Mappings
 
-### Basic
+As shown in the [Quick tutorial guide](@ref) it is possible to apply different mappings to the variables 
+using a simple syntax. 
+
+Alternatively, as shown in [Multiplying expressions](@ref), it is possible to define the mappings using 
+[`AbstractOperators.jl`](https://github.com/kul-forbes/ProximalAlgorithms.jl) and to apply them 
+to the variable (or expression) through multiplication.
+
+### Basic mappings
 ```@docs
 getindex
 reshape
 ```
 
-### DSP
+### DSP mappings
 ```@docs
 fft
 ifft
@@ -51,19 +67,22 @@ mimofilt
 zeropad
 ```
 
-### Finite differences
+### Finite differences mappings
 ```@docs
 finitediff
 variation
 ```
 
-### Nonlinear
+### Nonlinear mappings
 ```@docs
 sigmoid
 ```
 
 ## Utilities
 
+It is possible to access the variables, mappings and displacement of an expression. 
+Notice that these commands work also for the `Term`s described in [Functions and constraints](@ref).
+
 ```@docs
 variables
 operator
diff --git a/docs/src/functions.md b/docs/src/functions.md
@@ -1,5 +1,10 @@
 # Functions and constraints
 
+Once an expression is created it is possible to create the `Term`s defining the optimization problem. 
+
+These can consists of either [Smooth functions](@ref),  [Nonsmooth functions](@ref), [Inequality constraints](@ref) 
+or [Equality constraints](@ref).
+
 ## Smooth functions
 
 ```@docs
@@ -20,7 +25,7 @@ sumpositive
 hingeloss
 ```
 
-## Inequalities constraints
+## Inequality constraints
 
 ```@docs
 <=
@@ -34,12 +39,20 @@ hingeloss
 
 ## Smoothing
 
+Sometimes the optimization problem might involve only non-smooth terms which do not lead to efficient proximal mappings. It is possible to *smooth* this terms by means of the *Moreau envelope*.
+
 ```@docs
 smooth
 ```
 
 ## Duality
 
+In some cases it is more convenient to solve the *dual problem* instead of the primal problem. 
+
+It is possible to convert the primal problem into its dual form by means of the *convex conjugate*. 
+
+See the Total Variation demo for an example of such procedure.
+
 ```@docs
 conj
 ```
diff --git a/docs/src/index.md b/docs/src/index.md
@@ -14,14 +14,17 @@ three different packages:
 
 * [`ProximalAlgorithms.jl`](https://github.com/kul-forbes/ProximalAlgorithms.jl) is a library of proximal algorithms (aka splitting algorithms) solvers.
 
-`StructuredOptimization.jl` can handle large-scale convex and nonconvex problems with nonsmooth cost functions: see ? for a set of demos.
+`StructuredOptimization.jl` can handle large-scale convex and nonconvex problems with nonsmooth cost functions. It supports complex variables as well. See the demos and the [Quick tutorial guide](@ref).
+
+## Citing
+
+If you use `StructuredOptimization.jl` for published work, we encourage you to cite:
+
+* N. Antonello, L. Stella, P. Patrinos, T. van Waterschoot, “Proximal Gradient Algorithms: Applications in Signal Processing,” [arXiv:1803.01621](https://arxiv.org/abs/1803.01621) (2018).
 
 # Credits
 
 `StructuredOptimization.jl` is developed by
 [Lorenzo Stella](https://lostella.github.io) and
 [Niccolò Antonello](https://nantonel.github.io)
 at [KU Leuven, ESAT/Stadius](https://www.esat.kuleuven.be/stadius/).
-
-## Citing
-
diff --git a/docs/src/solvers.md b/docs/src/solvers.md
@@ -6,20 +6,69 @@
 @minimize
 ```
 
+!!! note "Problem warm-starting"
+
+    By default *warm-starting* is always enabled. 
+
+    For example, if two problems that utilize the same variables are solved consecutively, 
+    the second one will be automatically warm-started by the solution of the first one.
+
+    That is because the variables are always linked to their respective data vectors. 
+
+    If one wants to avoid this, the optimization variables needs to be manually re-initialized 
+    before solving the second problem e.g. to a vector of zeros: `~x .= 0.0`.
+
+
 ## Specifying solver and options
 
+As shown above it is possible to choose the type of algorithm and specify its options by creating a `Solver` object.
+
+Currently, the following algorithms are supported:
+
+* *Proximal Gradient (PG)* [[1]](http://www.mit.edu/~dimitrib/PTseng/papers/apgm.pdf), [[2]](http://epubs.siam.org/doi/abs/10.1137/080716542)
+* *Fast Proximal Gradient (FPG)* [[1]](http://www.mit.edu/~dimitrib/PTseng/papers/apgm.pdf), [[2]](http://epubs.siam.org/doi/abs/10.1137/080716542)
+* *ZeroFPR* [[3]](https://arxiv.org/abs/1606.06256)
+* *PANOC* [[4]](https://doi.org/10.1109/CDC.2017.8263933)
+
 ```@docs
 PG
 FPG
-PANOC
 ZeroFPR
+PANOC
 ```
 
 ## Build and solve
 
+The macro [`@minimize`](@ref) automatically parse and solve the problem. 
+
+An alternative syntax is given by the function [`problem`](@ref) and [`solve`](@ref).
+
 ```@docs
 problem
 solve
+```
+
+It is important to stress out that the `Solver` objects created using
+the functions above ([`PG`](@ref), [`FPG`](@ref), etc.)
+specify only the type of algorithm to be used together with its options. 
+
+The actual solver 
+(namely the one of [`ProximalAlgorithms.jl`](https://github.com/kul-forbes/ProximalAlgorithms.jl)) 
+is constructed altogether with the problem formulation. 
+
+The problem parsing procedure can be separated from the solver application using the functions [`build`](@ref) and [`solve!`](@ref).
+
+```@docs
 build
 solve!
 ```
+
+## Citations
+
+[[1]](http://www.mit.edu/~dimitrib/PTseng/papers/apgm.pdf) Tseng, *On Accelerated Proximal Gradient Methods for Convex-Concave Optimization* (2008).
+
+[[2]](http://epubs.siam.org/doi/abs/10.1137/080716542) Beck, Teboulle, *A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems*, SIAM Journal on Imaging Sciences, vol. 2, no. 1, pp. 183-202 (2009).
+
+[[3]](https://arxiv.org/abs/1606.06256) Themelis, Stella, Patrinos, *Forward-backward envelope for the sum of two nonconvex functions: Further properties and nonmonotone line-search algorithms*, arXiv:1606.06256 (2016).
+
+[[4]](https://doi.org/10.1109/CDC.2017.8263933) Stella, Themelis, Sopasakis, Patrinos, *A simple and efficient algorithm for nonlinear model predictive control*, 56th IEEE Conference on Decision and Control (2017).
diff --git a/docs/src/tutorial.md b/docs/src/tutorial.md
@@ -1,4 +1,4 @@
-# Quick Tutorial
+# Quick tutorial guide
 
 ## Standard problem formulation
 
@@ -18,9 +18,9 @@ The *least absolute shrinkage and selection operator* (LASSO) belongs to this cl
 \underset{ \mathbf{x} }{\text{minimize}} \ \tfrac{1}{2} \| \mathbf{A} \mathbf{x} - \mathbf{y} \|^2+ \lambda \| \mathbf{x} \|_1.
 ```
 
-Here the squared norm $\tfrac{1}{2} \| \mathbf{A} \mathbf{x} - \mathbf{y} \|^2$ is a *smooth* function while the $l_1$-norm is a *nonsmooth* function.
+Here the squared norm $\tfrac{1}{2} \| \mathbf{A} \mathbf{x} - \mathbf{y} \|^2$ is a *smooth* function $f$ wherelse the $l_1$-norm is a *nonsmooth* function $g$.
 
-This can be solved using `StructuredOptimization.jl` using only few lines of code:
+This problem can be solved using `StructuredOptimization.jl` using only few lines of code:
 
 ```julia
 julia> using StructuredOptimization
@@ -46,11 +46,11 @@ It is possible to access to the solution by typing `~x`.
 
 By default variables are initialized by `Array`s of zeros. 
 
-It is possible to set different initializations during construction `x = Variable( [1.; 0.; ...] )` or by assignement `~x .= [1.; 0.; ...]`.
+Different initializations can be set during construction `x = Variable( [1.; 0.; ...] )` or by assignement `~x .= [1.; 0.; ...]`.
 
 ## Constraint optimization
 
-Constraint optimization is also ecompassed by [Standard problem formulation](@ref): 
+Constraint optimization is also ecompassed by the [Standard problem formulation](@ref): 
 
 for a nonempty set $\mathcal{S}$ the constraint of 
 
@@ -61,7 +61,7 @@ for a nonempty set $\mathcal{S}$ the constraint of
 \end{align*}
 ```
 
-can be converted into an indicator function
+can be converted into an *indicator function*
 
 ```math
 g(\mathbf{x}) = \delta_{\mathcal{S}} (\mathbf{x}) =  \begin{cases}
@@ -78,17 +78,16 @@ For example, the non-negative deconvolution problem:
 
 ```math
 \begin{align*}
-\underset{ \mathbf{x} }{\text{minimize}} \ &  \tfrac{1}{2} \| \mathbf{x} * \mathbf{h} - \mathbf{y} \| \\
+\underset{ \mathbf{x} }{\text{minimize}} \ &  \tfrac{1}{2} \| \mathbf{x} * \mathbf{h} - \mathbf{y} \|^2 \\
 \text{subject to} \ & \mathbf{x} \geq 0
 \end{align*}
 ```
 
-where $*$ stands fof convoluton and $\mathbf{h}$ contains the taps of a finite impluse response.
-
-This problem be solved using the following line of code:
+where $*$ stands fof convoluton and $\mathbf{h}$ contains the taps of a finite impluse response, 
+can be solved using the following lines of code:
 
 ```julia
-julia> n = 10;
+julia> n = 10;                        # define problem size 
 
 julia> x = Variable(n);               # define variable
 
@@ -105,7 +104,11 @@ julia> @minimize ls(conv(x,h)-y) st x >= 0.
     `StructuredOptimization.jl` provides a set of functions that can be used to apply 
     specific operators to variables and create mathematical expression. 
     
-    The available functions can be found in [Operators](@ref).
+    The available functions can be found in [Mappings](@ref).
+
+    In general it is more convenient to use these functions instead of matrices, 
+    as these functions apply efficient algorithms for the forward and adjoint mappings leading to 
+    *matrix free optimization*.
 
 ## Using multiple variables
 
@@ -135,13 +138,15 @@ julia> @minimize ls(X1*X2-Y) st X1 >= 0., X2 >= 0.
 
 Currently `StructuredOptimization.jl` supports only *Proximal Gradient (aka Forward Backward) algorithms*, which require specific properties of the nonsmooth functions and costraint to be applicable.
 
+In particular, the nonsmooth functions must lead to an *efficiently computable proximal mapping*.
+
 If we express the nonsmooth function $g$ as the composition of 
 a function $\tilde{g}$ with a linear operator $A$: 
 ```math
 g(\mathbf{x}) =
 \tilde{g}(A \mathbf{x}) 
 ```
-than the problem can be solved when $g$ satisifies the following properties:
+then a proximal mapping of $g$ is efficiently computable if it satisifies the following properties:
 
 1. the mapping $A$ must be a *tight frame*  namely it must satisfy $A A^* = \mu Id$, where $\mu \geq 0$ and $A^*$ is the adjoint of $A$ and $Id$ is the identity operator.
 
@@ -184,3 +189,6 @@ julia> @minimize ls( A*x - y ) + λ*norm(x[1:div(n,2)], 1) st x[div(n,2)+1:n] >=
 ```
 as not the optimization variables $\mathbf{x}$ are partitioned into non-overlapping groups.
 
+!!! note 
+
+    When the problem is not accepted it might be still possible to solve it: see [Smoothing](@ref) and [Duality](@ref).
diff --git a/src/solvers/solvers_options.jl b/src/solvers/solvers_options.jl
@@ -12,7 +12,7 @@ export PG, FPG
 
 Creates an object `PG` containing the options of the Proximal Gradient solvers:
 
-  * `gamma`, stepsize (default: unspecified, determine automatically)
+  * `gamma`, stepsize (default: unspecified, determined automatically)
   * `maxit`, maximum number of iteration (default: `10000`)
   * `tol`, halting tolerance on the fixed-point residual (default: `1e-4`)
   * `adaptive`, adaptively adjust `gamma` (default: `false` if `gamma` is provided)
@@ -50,7 +50,7 @@ export ZeroFPR
 
 Creates an object `ZeroFPR` containing the options of the ZeroFPR solver:
 
-  * `gamma`, stepsize (default: unspecified, determine automatically)
+  * `gamma`, stepsize (default: unspecified, determined automatically)
   * `maxit`, maximum number of iteration (default: `10000`)
   * `tol`, halting tolerance on the fixed-point residual (default: `1e-4`)
   * `adaptive`, adaptively adjust `gamma` (default: `false` if `gamma` is provided)
@@ -79,7 +79,7 @@ export PANOC
 
 Creates an object `PANOC` containing the options of the PANOC solver:
 
-  * `gamma`, stepsize (default: unspecified, determine automatically)
+  * `gamma`, stepsize (default: unspecified, determined automatically)
   * `maxit`, maximum number of iteration (default: `10000`)
   * `tol`, halting tolerance on the fixed-point residual (default: `1e-4`)
   * `adaptive`, adaptively adjust `gamma` (default: `false` if `gamma` is provided)
diff --git a/src/syntax/expressions/addition.jl b/src/syntax/expressions/addition.jl
@@ -1,7 +1,7 @@
 import Base: +, -
 
 """
-`+(A::AbstractOperator, ex::AbstractExpression)`
+`+(ex1::AbstractExpression, ex2::AbstractExpression)`
 
 Add two expressions. 
 
@@ -20,7 +20,7 @@ julia> ex2 = randn(5,2)*z
 
 ```
 
-Notice that in order for two expressions to be added toghether their associate `AbstractOperator` 
+Notice that in order for two expressions to be added toghether their associated `AbstractOperator` 
 must have the same codomain:
 
 ```julia
@@ -173,8 +173,8 @@ julia> ex = x+4
 
 ```
 
-Notice that in order to add an array to `ex`, `b` must belong to  
-of the associate `AbstractOperator` of `ex`. 
+Notice that in order to add an array to `ex`, `b` must belong to the codomain 
+of the associated `AbstractOperator` of `ex`. 
 
 ```julia
 julia> b = randn(10);
diff --git a/src/syntax/expressions/multiplication.jl b/src/syntax/expressions/multiplication.jl
diff --git a/src/syntax/expressions/utils.jl b/src/syntax/expressions/utils.jl
diff --git a/src/syntax/terms/proximalOperators_bind.jl b/src/syntax/terms/proximalOperators_bind.jl
