Detailed simconfig settings

[hakoda_723]

Hi all, we implemented a simulator that switches between several learning situations with simconfig values. By selecting these values during training, we intend to have a single agent behave in multiple situations in parallel.

Is it possible to implement in inkling so that the simconfig value cycles through each episode like 0, 1, 2, 0, 1, 2,... , instead of a random selection like below?

In addition, how can I set the simconfig value according to the performance of the automatic assessment?

Details are described in sapplementary material.

[#2 · Accepted solution]

Hi @hakoda_723, thanks for posting your question.

There's currently no way to specify the order in which episodes cycle through configuration values. You can define a curriculum that contains multiple lessons, and each lesson could focus on one (or a subset of) config values, but that's a bit different that what you're looking to do.

You will probably need to implement this cycling within your simulator. For example, add a mode that is enabled through a config value. When in the "cycling" mode, the sim could maintain its own episode count and choose an appropriate value such that it cycles through all of the learning situations in turn.

[#3]

@hakoda_723Eric's suggestion was great and I wanted to add two points for you or any future readers:

1. At least for AnyLogic simulators, each new run completely wipes out the state of the previous, so Eric's suggestion (to have the simulator keep track of the count) doesn't appear to be feasible. However, as long as the same container is being used to run the episodes, the same JVM is being used. A consequence is that any static variables keep their status. So if you have a static variable for a run counter and increment that at the start of each run, it should work as intended. Though if you only want this for testing, you'll need some other flag to indicate whether to reference it or not.

2.  A possibly simpler approach could be just to generate a configuration for the number of runs you want to do, download it, then load it into Python (or your preferred language) to edit the JSON file. Then you can reupload and use that!