Training simulations for Bonsai
Training simulations replicate real-world systems to provide an authentic training environment for Bonsai brains. You can add simulations from popular simulator software solutions or use the Simulator API to integrate custom Python simulators.
icons highlighted.
Training simulations model real-world processes and change state as the brain applies actions. Robotics, industrial automation, supply chain logistics, and structural engineering are all domains that use simulations to model the behavior of complex systems.
Bonsai uses simulations and Deep Reinforcement Learning (DRL) to train brains. Training tasks can be as simple as "keep this pole upright" or as complex as "learn to walk."
Generally speaking, any simulation that has a defined start state, iterates over time, and responds to external actions can integrate with Bonsai. But simulations that work well with Bonsai have the following characteristics:
- An appropriate level of fidelity so that strategies developed against the simulation are likely to work well in the real world.
- Useful visualization and data output while controlled by the brain for real-time assessment during training.
- A well-defined environment state that is accessible at each step of the simulation.
- A customizable start state so the brain can learn from a wide array of conditions.
- A set of discrete actions the brain can take to affect the state. For example: move a cart one step on a track, adjust a temperature by 1° Celsius.
- The ability to determine when the system gets into a state where further progress is impossible (a failure or invalid state). For example: the cart runs off the track, the current temperature exceeds a quality threshold.
- The ability to determine when the system reaches a success state. For example, a pole balances for a specific amount of time, the generated material passes QC requirements.
Determining the right level of fidelity for a simulation depends on:
- the precision required for individual actions.
- the probability that the AI could recover from an imperfect action in the real world.
For example, AI could compensate for an unexpectedly wide turn caused by a real-world car turning 1 km faster than the simulated car the AI trained with. But, if that same car regularly turns 10 km faster than the simulated car, the car could flip over or run off the road.
When considering your simulation approach, it may be helpful to look for people in your organization who have worked with simulation software before. Simulations originally created for other purposes can often be enhanced to work with the Bonsai training engine. Look for existing simulations with one or more of the following characteristics:
- Simulations used to train human operators.
- Simulations regularly used in conjunction with production systems.
- Simulations with well-defined benchmarks for accuracy and desired outcomes.