Collecting Data

After your simulation has completed, you probably want to get some data back. FLAME GPU provides two methods of achieving this: Logging which can reduce a large simulation state down to the most important data points, and export of the entire simulation state.

Note

It is not currently possible to export environment directed graphs via logging or JSON/XML state files. Environment directed graphs can only be imported and exported during host functions using importGraph() and exportGraph(), which utilises a common JSON graph storage format.

Logging

As Simulations are likely to involve thousands to millions of agents, logging the full simulation state would produce large amounts of output most of which is not required. As such, FLAME GPU provides the ability to specify a collection of population level reductions and environment properties to be logged each step or at simulation exit.

Configuring Data to be Logged

To define a logging config, you should create either a LoggingConfig or StepLoggingConfig, passing your ModelDescription to the constructor. These two classes share the same interface for specifying data to be logged, however StepLoggingConfig additionally allows you to call setFrequency() to adjust how often steps are logged (default 1, every step).

Environment properties are logged with logEnvironment(), specifying the property’s name. Unlike most FLAME GPU variable and property methods, the type does not need to be specified here.

Agent data is logged according to agent state, so agents with multiple states must have the config specified for each state required to be logged. The total number of agents in the state can be logged with logCount(). Or variable reductions can be logged with; logMean(), logStandardDev(), logMin(), logMax(), logSum() specifying the variable’s name and type (in the same form as used throughout the API).

Once setup, the logging configs are passed to the CUDASimulation using setStepLog() and setExitLog().

flamegpu::ModelDescription model("example model");

// Fully define the model
...

// Specify the desired LoggingConfig or StepLoggingConfig
flamegpu::StepLoggingConfig step_log_cfg(model);
{
    // Log every step (not available to LoggingConfig, for exit logs)
    step_log_cfg.setFrequency(1);
    // Include the environment property 'env_prop' in the logged data
    step_log_cfg.logEnvironment("env_prop");
    // Include the current number of 'boid' agents, within the 'default' state
    step_log_cfg.agent("boid").logCount();
    // Include the current number of 'boid' agents, within the 'alive' state
    step_log_cfg.agent("boid", "alive").logCount();
    // Include the mean of the boid agents population's variable 'speed', within the default state
    step_log_cfg.agent("boid").logMean<float>("speed");
    // Include the standard deviation of the boid agent population's variable 'speed', within the default state
    step_log_cfg.agent("boid").logStandardDev<float>("speed");
    // Include the min and max of the boid agent population's variable 'speed', within the default state
    step_log_cfg.agent("boid").logMin<float>("speed");
    step_log_cfg.agent("boid").logMax<float>("speed");
    // Include the sum of the boid agent population's variable 'health', within the 'alive' state
    step_log_cfg.agent("boid", "alive").logSum<int>("health");
}

// Create the CUDASimulation instance
flamegpu::CUDASimulation cuda_sim(model);

// Attach the logging config/s
cuda_sim.setStepLog(step_log_cfg);
// cuda_sim.setExitLog(exit_log_cfg);

// Run the simulation as normal
cuda_sim.simulate();

Accessing Collected Data

After configuring a CUDASimulation to use specific logging configs, and executing the simulation, the log can be accessed via code using getRunLog(). This returns a RunLog which contains the step and exit log data that was requested.

Performance data is always attached to the requested logs, so can be accessed if required.

// Attach the logging config/s
cuda_sim.setStepLog(step_log_cfg);
cuda_sim.setExitLog(exit_log_cfg);

// Run the simulation as normal
cuda_sim.simulate();

// Fetch the logged data
flamegpu::RunLog run_log = cuda_sim.getRunLog();

// Get the random seed used
uint64_t rng_seed = run_log.getRandomSeed();
// Get the step logging frequency
unsigned int step_log_freqency = run_log.getStepLogFrequency();

// Access the step and exit log data
// The step and exit logs will be empty, if a respective logging config was not specified.
flamegpu::LogFrame exit_log = run_log.getExitLog();
std::list<flamegpu::LogFrame> step_log = run_log.getStepLog();

// Iterate the step log and print some information to console
for (auto &log:step_log) {
    // Get the step index
    unsigned int step_count = log.getStepCount();
    // Get a logged environment property
    int env_prop = log.getEnvironmentProperty<int>("env_prop");
    // Get logged boid agent property reduction data, from the default state
    unsigned int agent_count = log.getAgent("boid").getCount();
    // Reduce operators upcast the return type to the most suitable to not lose data
    double agent_speed_mean = log.getAgent("boid").getMean("speed");
    // Print data to console
    printf("#%u: %u, %f\n", step+count, agent_count, agent_speed_mean);
}

Writing Collected Data to File

Instead of processing logged data at runtime, you can store it to file for post-processing at a later time.

Normally you would handle this via the Simulation::Config as detailed in the earlier section. However, you can also call exportLog() on the CUDASimulation, to manually trigger the export.

// Attach the logging config/s
cuda_sim.setStepLog(step_log_cfg);
cuda_sim.setExitLog(exit_log_cfg);

// Run the simulation as normal
cuda_sim.simulate();

// Export the logged data to file
cuda_sim.exportLog(
  "log.json", // The file to output (must end '.json' or '.xml')
  true,       // Whether the step log should be included in the log file
  true,       // Whether the exit log should be included in the log file
  true,       // Whether the step time should be included in the log file (treated as false if step log not included)
  true,       // Whether the simulation time should be included in the log file (treated as false if exit log not included)
  false       // Whether the log file should be minified or not
);

Accessing the Complete Agent State

In some limited cases, you may want to directly access a full agent population. This can only be achieved in code, either by directly accessing the agent data or manually triggering the export to file.

Similar to specifying an initial agent population, you can fetch an agent state population to an AgentVector.

flamegpu::ModelDescription model("example model");
flamegpu::AgentDescription boid_agent = model.newAgent("boid");

// Fully define the model & setup the CUDASimulation
...

// Run the simulation as normal
// step() could also be used to access the agent state, on a per step basis
cuda_sim.simulate();

// Copy the boid agent data, from the default state, to an agent vector
flamegpu::AgentVector out_pop(boid_agent);
cuda_sim.getPopulationData(out_pop);

// Iterate the agents, and print their speed
for (flamegpu::AgentVector::Agent &boid : out_pop) {
    printf("Speed: %f\n", boid.getVariable<float>("speed"));
}

Alternatively, exportData() can be called to export the full simulation state to file (all agent variables and environment properties).

flamegpu::ModelDescription model("example model");
flamegpu::AgentDescription boid_agent = model.newAgent("boid");

// Fully define the model & setup the CUDASimulation
...

// Run the simulation as normal
// step() could also be used to access the agent state, on a per step basis
cuda_sim.simulate();

// Log the simulation state to JSON/XML file
cuda_sim.exportData("end.json");

Additional Notes

At the time of writing it is not possible to log or export Environment Macro Properties, doing so would require manually outputting them via an init, step or exit function.