This article proposes a consistent and general approach to train physics-augmented neural networks with observable data to enrich and represent nonlinear history-dependent material behaviors in terms of both state equations and evolution laws. In this learning strategy consistent with thermodynamics, the constitutive model is expressed using two potentials (free energy and dissipation potential) which are represented by input-convex neural networks, thus automatically satisfying the principles of thermodynamics. The neural network is trained thanks to an unsupervised procedure that does not rely on strain–stress pairs but needs only partial strain or displacement measurements inside the structure, moreover with uncertain boundary conditions. This method is based on the minimization of the modified Constitutive Relation Error functional, and it extends previous works on this error measure for neural networks to the case of history-dependent behaviors, which requires to design a specific minimization procedure. Given that neural networks for typical structural health monitoring applications often need to be trained online, there is here a significant emphasis placed on automatically and adaptively tuning crucial hyperparameters such as learning rate or weighting between losses. The method is evaluated on elastoplastic and elastoviscoplastic test cases with synthetic data collected from optic fiber or full-field measurements. It is shown that the method can properly learn hidden behaviors, achieves high robustness to noise level, and low sensitivity to user-defined hyperparameters.