Development of a model-free Hamiltonian tracking optimal control algorithm

In this study, a novel algorithm has been developed to solve a trajectory optimization problem of a model-free black box dynamical system. The proposed algorithm does not need an explicit dynamic model of the system but computes partial derivatives of the dynamic function numerically from the time series data of observation to estimate the adjoint variable and the Hamiltonian. The additional necessary conditions for optimality, constant Hamiltonian over time span, are used as the tracking condition to find an optimal trajectory. A candidate optimal trajectory is searched by the Legendre transformation which interprets the geometric information of the current control trajectory on the Lagrangian surface. The implication of this approach is the elimination of the need for the dynamic model or the system identification process as we only derive necessary partial derivatives out of current observations. This enables us to find a near optimal trajectory quickly without the explicit dynamic model or the full system identification process. The estimated Hamiltonian approach is verified first with several problems whose dynamic models are known. After then, the model-free algorithm is applied for several problems where the dynamics are still unclear. First case is real world applications where the observation data is obtained by experiments or from historical record. These applications include a recent hot manufacturing process called Field Assisted Sintering Technology (FAST) and a socio-economic policy problem of water usage management by price controls. In this case, approximated dynamic models based on collected empirical data are used for the simulated iterations to validate the effectivenesv of the proposed algorithm. The proposed algorithm only use the observation output and shows iterative candidate searching history which converges toward an exact solution or a certain trajectory with decreasing total cost. Second case is a simulated feedback con- trol algorithm called distributed arrival time control (DATC) which is well known for its fast searching capability of a near optimal queueing sequence that minimizes tardiness error from due dates. The dynamics of DATC is highly non-linear and discontinuous due to a queueing effect in the sequence when the due dates cannot be fully satisfied simultaneously. The proposed algorithm is applied to this DATC algorithm as a reiforcement module which improves optimal search effort. The effectiveness and limit of the proposed algorithm is discussed in the perspective of true optimal searching capability. Finally, the implication of the proposed method and the direction of future research are discussed as well as plausible application areas.