-------------------------------------------
lines 6-324 of file: src/optimize_fixed.cpp
-------------------------------------------

{xrst_begin optimize_fixed}
{xrst_spell
  iter
  nlp
  struct
}

Optimize Fixed Effects
######################

Syntax
******
*solution*  =

| *mixed_object* . ``optimize_fixed`` (
| |tab| *fixed_ipopt_options* ,
| |tab| *random_ipopt_options* ,
| |tab| *fixed_lower* ,
| |tab| *fixed_upper* ,
| |tab| *fix_constraint_lower* ,
| |tab| *fix_constraint_upper* ,
| |tab| *fixed_scale* ,
| |tab| *fixed_in* ,
| |tab| *random_lower* ,
| |tab| *random_upper* ,
| |tab| *random_in* ,
| |tab| *warm_start*
| )

Purpose
*******
This routine maximizes the fixed effects objective
:ref:`L(theta)<theory@Objective@Fixed Effects Objective, L(theta)>` .

inf
***
The value ``inf`` below refers to

   ``std::numeric_limits<double>::infinity`` ()

mixed_object
************
We use :ref:`derived_ctor@mixed_object`
to denote an object of a class that is
derived from the ``cppad_mixed`` base class.

fixed_ipopt_options
*******************
This argument has prototype

   ``const std::string&`` *fixed_ipopt_options*

and is the :ref:`ipopt_options-name` for optimizing the fixed effects
with the following qualifications:

derivative_test
===============
If *derivative_test* is ``none`` ,
no derivative testing is done.
If it is ``first-order`` ,
only first order derivatives are tested.
If it is ``second-order`` (``only-second-order`` )
second derivatives (and first derivatives) are tested.
In these two cases
:ref:`derived_ctor@quasi_fixed` must be ``false`` .
If *derivative_test* is ``adaptive`` ,
a special ``cppad_mixed`` adaptive step size method is
used to test first order derivatives.
If it is ``trace-adaptive`` ,
the adaptive step size results are traced on standard output.

hessian_approximation
=====================
If *quasi_fixed* is true (false),
*hessian_approximation* will be set to
``limit-memory`` (``exact`` ).
If it is also set in *fixed_ipopt_options* , it must have this value.

max_iter
========
If *max_iter* == ``-1`` in *fixed_ipopt_options* ,
*solution* . ``fixed_opt`` == *fixed_in* .
In addition, Ipopt is run with *max_iter*  = 0 and the return status
``Ipopt::Maximum_Iterations_Exceeded`` is consider normal; i.e.,
does not generate a warning or error message.
Furthermore, the fixed effects optimization will return immediately
(not try to backup and recover) if an error occur during evaluation of
the fixed effects objective, constraints, or their derivatives.
If *max_iter*  == 0 in the options,
it may be that *solution* . ``fixed_opt`` != *fixed_in*
(Ipopt moves non-equality constraints to the interior of the constraint).
(this is the only difference between ``-1`` and ``0`` ).

accept_after_max_steps
======================
The default value for *accept_after_max_steps* is ``-1``
(no limit).
This is the maximum number of backtracking steps to take before
accepting a line search point; see
:ref:`ipopt_trace@ls` is the ipopt tracing documentation.

nlp_scaling_method
==================
When optimizing the fixed effects,
the objective and the constraint functions are automatically scaled
by ``cppad_mixed`` .
It is an error for the user to specify this option in
*fixed_ipopt_options* .

random_ipopt_options
********************
This argument has prototype

   ``const std::string&`` *random_ipopt_options*

and is the :ref:`ipopt_options-name` for optimizing the random effects.

fixed_lower
***********
This argument has prototype

   ``const CppAD::vector<double>&`` *fixed_lower*

It has size :ref:`derived_ctor@n_fixed` and
specifies the lower limits for the
:ref:`fixed effects<problem@Notation@Fixed Effects, theta>` .
Note that minus infinity is used for no lower limit.

fixed_upper
***********
This argument has prototype

   ``const CppAD::vector<double>&`` *fixed_upper*

It has size *n_fixed* and specifies the upper limits for the
:ref:`fixed effects<problem@Notation@Fixed Effects, theta>` .
Note that plus infinity is used for no upper limit.

fix_constraint_lower
********************
This argument has prototype

   ``const CppAD::vector<double>&`` *fix_constraint_lower*

it has size *n_fixed* and specifies the lower limits for the
:ref:`fixed constraints<fix_constraint-name>` .
Note that minus infinity is used for no lower limit.

fix_constraint_upper
********************
This argument has prototype

   ``const CppAD::vector<double>&`` *fix_constraint_upper*

it specifies the upper limits for the
:ref:`fixed constraints<fix_constraint-name>` .
Note that plus infinity is used for no upper limit.

fixed_scale
***********
This argument has prototype

   ``const CppAD::vector<double>&`` *fixed_scale*

The fixed effect objective and constraint functions are multiplied by a
scale factor so that their derivatives are near one at *fixed_scale* .
This makes the Ipopt tolerance be relative to the derivatives at
*fixed_scale* .
It must hold for each *j* that

   *fixed_lower* [ *j* ] <= *fixed_scale* [ *j* ] <= *fixed_upper* [ *j* ]

Partial derivatives with respect to components for which

   *fixed_lower* [ *j* ] == *fixed_upper* [ *j* ]

are not included in this scaling.
Note that you can continue an optimization with the same scaling
by setting

   *fixed_in* = *solution* . ``fixed_opt``

and the re-running the optimization.
Also note that scaling the fixed effects is not done by ``cppad_mixed``
and should be done by the users program when it is useful.

fixed_in
********
This argument has prototype

   ``const CppAD::vector<double>&`` *fixed_in*

It specifies the initial value for the
:ref:`fixed effects<problem@Notation@Fixed Effects, theta>`
vector :math:`\theta` during the optimization process.
It must hold for each *j* that

   *fixed_lower* [ *j* ] <= *fixed_in* [ *j* ] <= *fixed_upper* [ *j* ]

random_lower
************
This argument has prototype

   ``const CppAD::vector<double>&`` *random_lower*

It must have size equal to
:ref:`derived_ctor@n_random` and
specifies the lower limits for the optimization of the
:ref:`random effects<problem@Notation@Random Effects, u>`
vector :math:`u`.
This may be useful to keep the random effects
out of regions of numerical instability.
On the other hand, the calculation of the
:ref:`theory@Derivative of Optimal Random Effects`
:math:`\hat{u}_\theta ( \theta )` will not be correct when these constraints
are active (and this could have adverse effects on the optimization).
The value minus infinity can be used to specify no lower limit.

random_upper
************
This argument has prototype

   ``const CppAD::vector<double>&`` *random_upper*

It must have size equal to
:ref:`derived_ctor@n_random` and
specifies the upper limits for the optimization of the random effect.
The value plus infinity can be used to specify no lower limit.

random_in
*********
This argument has prototype

   ``const CppAD::vector<double>&`` *random_in*

It must have size equal to
:ref:`derived_ctor@n_random` and
specifies the initial value used for the optimization of the
:ref:`random effects<problem@Notation@Random Effects, u>`
vector :math:`u`.
It must hold that

   *random_lower* [ *i* ] <= *random_in* [ *i* ] <= *random_upper* [ *i* ]

for each valid index *i* .

warm_start
**********
This argument is optional and has prototype

   ``const warm_start_struct&`` *warm_start*

It is an error for the user to specify *warm_start_init_point* in
*fixed_ipopt_options* .

No Warm Start
=============
If the size of *warm_start* . ``x_info`` is zero,
there is no warm start information.
This is the same as when the argument is not present.
In this case, the ipopt *warm_start_init_point* option will be set to
``no`` .

Warm Start
==========
If the size of *warm_start* . ``x_info`` is non-zero,
*warm_start* must is equal the
:ref:`fixed_solution@warm_start` field in a
fixed effects solution returned by a previous call to ``optimized_fixed`` .
This can be used to continue a fit when the maximum number of iterations
is reached or when the tolerance for the fixed or random effects is changed.

#. The ipopt *warm_start_init_point* options will be set to ``yes`` .
#. The ipopt *mu_strategy* options will be set to ``monotone`` .
#. The *fixed_scale* and *fixed_in* arguments are not used
   during a warm start optimization.
#. Any derivative test specified in *fixed_ipopt_options*
   will not be passed onto Ipopt; i.e.,
   no derivative testing is done during a warm start.

Example
=======
see :ref:`warm_start.cpp-name`

solution
********
The return value has prototype

   ``CppAD::mixed::fixed_solution`` *solution*

It is the solution (obtained by optimization) of the
fixed effects vector and its Lagrange multipliers; see
:ref:`fixed_solution-name` .

Laplace Approximation
*********************
The :ref:`theory-name` for the
Laplace approximation optimization only includes the case where
the :ref:`random likelihood<ran_likelihood-name>` is smooth.

{xrst_comment ipopt_options is also used by optimize_random}
{xrst_toc_hidden
   example/user/optimize_fixed.cpp
   xrst/ipopt_options.xrst
   xrst/ipopt_trace.xrst
}
Example
*******
The file :ref:`optimize_fixed.cpp-name` contains an example
and test of this procedure.
It returns true, if the test passes, and false otherwise.

ipopt_fixed
***********
The  class ``ipopt_fixed`` is used by ``optimize_fixed``
to optimize the fixed effects.
It's specifications are not part of the :ref:`cppad_mixed-name` public interface.

{xrst_end optimize_fixed}