Exercise 4.5: Show the equivalence of 3 convex problem formations

% From Boyd & Vandenberghe, "Convex Optimization"
% Joëlle Skaf - 08/17/05
%
% Shows the equivalence of the following 3 problems:
% 1) Robust least-squares problem
%           minimize    sum_{i=1}^{m} phi(a_i'*x - bi)
%    where phi(u) = u^2             for |u| <= M
%                   M(2|u| - M)     for |u| >  M
% 2) Least-squares with variable weights
%           minimize    sum_{i=1}^{m} (a_i'*x - bi)^2/(w_i+1) + M^2*1'*w
%               s.t.    w >= 0
% 3) Quadratic program
%           minimize    sum_{i=1}^{m} (u_i^2 + 2*M*v_i)
%               s.t.    -u - v <= Ax - b <= u + v
%                       0 <= u <= M*1
%                       v >= 0

% Generate input data
randn('state',0);
m = 16; n = 8;
A = randn(m,n);
b = randn(m,1);
M = 2;

% (a) robust least-squares problem
disp('Computing the solution of the robust least-squares problem...');
cvx_begin
    variable x1(n)
    minimize( sum(huber(A*x1-b,M)) )
cvx_end

% (b)least-squares problem with variable weights
disp('Computing the solution of the least-squares problem with variable weights...');
cvx_begin
    variable x2(n)
    variable w(m)
    minimize( sum(quad_over_lin(diag(A*x2-b),w'+1)) + M^2*ones(1,m)*w)
    w >= 0;
cvx_end

% (c) quadratic program
disp('Computing the solution of the quadratic program...');
cvx_begin
    variable x3(n)
    variable u(m)
    variable v(m)
    minimize( sum(square(u) +  2*M*v) )
    A*x3 - b <= u + v;
    A*x3 - b >= -u - v;
    u >= 0;
    u <= M;
    v >= 0;
cvx_end

% Display results
disp('------------------------------------------------------------------------');
disp('The optimal solutions for problem formulations 1, 2 and 3 are given');
disp('respectively as follows (per column): ');
[x1 x2 x3]
Computing the solution of the robust least-squares problem...
 
Calling SDPT3: 112 variables, 56 equality constraints
   For improved efficiency, SDPT3 is solving the dual problem.
------------------------------------------------------------

 num. of constraints = 56
 dim. of sdp    var  = 32,   num. of sdp  blk  = 16
 dim. of socp   var  = 32,   num. of socp blk  = 16
 dim. of linear var  = 32
*******************************************************************
   SDPT3: Infeasible path-following algorithms
*******************************************************************
 version  predcorr  gam  expon  scale_data
   HKM      1      0.000   1        0    
it pstep dstep pinfeas dinfeas  gap      mean(obj)   cputime
-------------------------------------------------------------------
 0|0.000|0.000|7.4e+00|5.3e+00|4.1e+03| 2.130193e+02| 0:0:00| chol  1  1 
 1|1.000|1.000|2.2e-05|8.4e-02|5.8e+02|-9.599552e+01| 0:0:00| chol  1  1 
 2|1.000|1.000|4.0e-06|8.4e-03|1.3e+02|-1.195969e+01| 0:0:00| chol  1  1 
 3|1.000|1.000|8.2e-07|8.4e-04|2.8e+01|-5.394008e+00| 0:0:00| chol  1  1 
 4|0.972|0.967|1.9e-07|1.1e-04|1.8e+00|-4.274103e+00| 0:0:00| chol  1  1 
 5|0.966|0.942|1.3e-08|1.4e-05|2.4e-01|-4.221792e+00| 0:0:00| chol  1  1 
 6|1.000|1.000|3.0e-09|8.4e-07|4.4e-02|-4.209702e+00| 0:0:00| chol  1  1 
 7|0.988|0.988|8.7e-10|9.4e-08|5.2e-04|-4.209695e+00| 0:0:00| chol  1  1 
 8|0.988|0.989|1.2e-10|1.2e-09|6.0e-06|-4.209705e+00| 0:0:00| chol  1  1 
 9|0.998|0.999|1.7e-11|2.6e-11|8.6e-08|-4.209705e+00| 0:0:00|
  stop: max(relative gap, infeasibilities) < 1.49e-08
-------------------------------------------------------------------
 number of iterations   =  9
 primal objective value = -4.20970517e+00
 dual   objective value = -4.20970525e+00
 gap := trace(XZ)       = 8.59e-08
 relative gap           = 9.12e-09
 actual relative gap    = 8.86e-09
 rel. primal infeas     = 1.66e-11
 rel. dual   infeas     = 2.55e-11
 norm(X), norm(y), norm(Z) = 1.5e+01, 2.9e+00, 8.7e+00
 norm(A), norm(b), norm(C) = 2.3e+01, 5.0e+00, 1.1e+01
 Total CPU time (secs)  = 0.4  
 CPU time per iteration = 0.0  
 termination code       =  0
 DIMACS: 4.1e-11  0.0e+00  9.1e-11  0.0e+00  8.9e-09  9.1e-09
-------------------------------------------------------------------
------------------------------------------------------------
Status: Solved
Optimal value (cvx_optval): +4.20971
Computing the solution of the least-squares problem with variable weights...
 
Calling SDPT3: 304 variables, 40 equality constraints
   For improved efficiency, SDPT3 is solving the dual problem.
------------------------------------------------------------

 num. of constraints = 40
 dim. of socp   var  = 288,   num. of socp blk  = 16
 dim. of linear var  = 16
*******************************************************************
   SDPT3: Infeasible path-following algorithms
*******************************************************************
 version  predcorr  gam  expon  scale_data
    NT      1      0.000   1        0    
it pstep dstep pinfeas dinfeas  gap      mean(obj)   cputime
-------------------------------------------------------------------
 0|0.000|0.000|2.9e+00|4.8e+00|1.6e+03| 0.000000e+00| 0:0:00| chol  1  1 
 1|1.000|1.000|1.3e-06|8.9e-02|2.3e+02|-1.108000e+02| 0:0:00| chol  1  1 
 2|1.000|0.912|1.7e-07|1.6e-02|1.9e+01|-2.011103e+01| 0:0:00| chol  1  1 
 3|0.958|0.972|4.4e-08|1.3e-03|7.0e-01|-2.014984e+01| 0:0:00| chol  1  1 
 4|0.985|0.983|1.7e-08|1.1e-04|1.1e-02|-2.020523e+01| 0:0:00| chol  1  1 
 5|0.969|0.978|1.3e-08|1.1e-05|2.9e-04|-2.020921e+01| 0:0:00| chol  1  1 
 6|0.909|0.988|4.3e-09|1.4e-07|1.7e-05|-2.020970e+01| 0:0:00| chol  1  1 
 7|0.903|0.987|8.6e-10|2.6e-09|1.6e-06|-2.020970e+01| 0:0:00| chol  2  2 
 8|0.795|0.924|2.5e-10|3.7e-10|3.0e-07|-2.020971e+01| 0:0:00|
  stop: max(relative gap, infeasibilities) < 1.49e-08
-------------------------------------------------------------------
 number of iterations   =  8
 primal objective value = -2.02097050e+01
 dual   objective value = -2.02097053e+01
 gap := trace(XZ)       = 2.96e-07
 relative gap           = 7.15e-09
 actual relative gap    = 6.30e-09
 rel. primal infeas     = 2.53e-10
 rel. dual   infeas     = 3.72e-10
 norm(X), norm(y), norm(Z) = 1.7e+01, 2.9e+00, 3.7e+00
 norm(A), norm(b), norm(C) = 1.4e+01, 1.5e+01, 6.4e+00
 Total CPU time (secs)  = 0.2  
 CPU time per iteration = 0.0  
 termination code       =  0
 DIMACS: 9.8e-10  0.0e+00  9.5e-10  0.0e+00  6.3e-09  7.1e-09
-------------------------------------------------------------------
------------------------------------------------------------
Status: Solved
Optimal value (cvx_optval): +4.20971
Computing the solution of the quadratic program...
 
Calling SDPT3: 128 variables, 56 equality constraints
   For improved efficiency, SDPT3 is solving the dual problem.
------------------------------------------------------------

 num. of constraints = 56
 dim. of sdp    var  = 32,   num. of sdp  blk  = 16
 dim. of linear var  = 80
*******************************************************************
   SDPT3: Infeasible path-following algorithms
*******************************************************************
 version  predcorr  gam  expon  scale_data
   HKM      1      0.000   1        0    
it pstep dstep pinfeas dinfeas  gap      mean(obj)   cputime
-------------------------------------------------------------------
 0|0.000|0.000|1.4e+01|8.1e+00|1.6e+04| 3.662167e+02| 0:0:00| chol  1  1 
 1|0.945|1.000|7.7e-01|9.4e-02|1.5e+03|-3.969715e+01| 0:0:00| chol  1  1 
 2|0.988|1.000|9.3e-03|9.4e-03|1.4e+02|-2.593681e+01| 0:0:00| chol  1  1 
 3|1.000|0.858|9.6e-07|4.0e-03|2.4e+01|-5.097221e+00| 0:0:00| chol  1  1 
 4|1.000|1.000|1.9e-08|9.4e-05|9.8e+00|-3.774663e+00| 0:0:00| chol  1  1 
 5|0.925|1.000|1.8e-09|9.4e-06|1.4e+00|-3.997147e+00| 0:0:00| chol  1  1 
 6|1.000|1.000|4.1e-10|9.4e-07|6.3e-01|-4.170753e+00| 0:0:00| chol  1  1 
 7|0.945|0.941|1.6e-10|1.4e-07|4.5e-02|-4.206977e+00| 0:0:00| chol  1  1 
 8|0.968|0.989|5.7e-11|1.1e-08|1.8e-03|-4.209354e+00| 0:0:00| chol  1  1 
 9|0.976|0.985|1.3e-10|1.1e-09|3.8e-05|-4.209695e+00| 0:0:00| chol  1  1 
10|1.000|1.000|1.5e-13|1.7e-11|1.2e-06|-4.209705e+00| 0:0:00| chol  1  1 
11|1.000|1.000|2.7e-12|1.0e-12|3.2e-08|-4.209705e+00| 0:0:00|
  stop: max(relative gap, infeasibilities) < 1.49e-08
-------------------------------------------------------------------
 number of iterations   = 11
 primal objective value = -4.20970519e+00
 dual   objective value = -4.20970523e+00
 gap := trace(XZ)       = 3.17e-08
 relative gap           = 3.37e-09
 actual relative gap    = 3.36e-09
 rel. primal infeas     = 2.65e-12
 rel. dual   infeas     = 1.00e-12
 norm(X), norm(y), norm(Z) = 1.4e+01, 2.9e+00, 9.4e+00
 norm(A), norm(b), norm(C) = 2.0e+01, 1.7e+01, 1.1e+01
 Total CPU time (secs)  = 0.3  
 CPU time per iteration = 0.0  
 termination code       =  0
 DIMACS: 9.3e-12  0.0e+00  3.8e-12  0.0e+00  3.4e-09  3.4e-09
-------------------------------------------------------------------
------------------------------------------------------------
Status: Solved
Optimal value (cvx_optval): +4.20971
------------------------------------------------------------------------
The optimal solutions for problem formulations 1, 2 and 3 are given
respectively as follows (per column): 

ans =

    0.3888    0.3888    0.3888
    0.1262    0.1262    0.1262
   -0.3337   -0.3337   -0.3337
    0.1326    0.1325    0.1326
    0.5500    0.5500    0.5500
    0.3526    0.3526    0.3526
   -0.6562   -0.6562   -0.6562
    0.8309    0.8309    0.8309