Preface

This is the first edition of the project documentation, compiled in a form of a short book. It's still work in progress, which means a lot of errors and unfinished pages. If you've spotted some silly error, or want to add a paragraph somewhere, please create an issue on github.

What does this book contain?

It contains a compilation of easy to read documents describing the project, it's various goals and elements.

It also covers things that are relevant to the process of creating simulation models and handling data using the proposed API.

Who is it intended for?

It's for anyone who wishes to understand the design behind outcome simulations, how they are run, how the data and simulation models (called collectively content) can be created and handled, among other things. Getting a good grip on the conceptual make up of the architecture is recommended before getting into creating content.

It's also for anyone who doesn't want to dive into specifics of how things are done "behind the scenes", but instead is interested in learning about creating content and learning how to use the provided tools. It's okay to skip the boring chapters and get into running simulations right away - one can always revisit certain chapters as you encounter problems later along the way.

How is it written - aiming for accessibility

Documentation for this project is meant to be easy to read and understand by anyone who doesn't have any prior experience with computer programming or game modding.

That said, this first edition is not kept up to this high standard of accessibility. It still contains things that can be considered to require some prior experience/knowledge related to the categories mentioned above. The goal for the next editions will be to further refine the documents in this book to:

  • reduce usage of highly "industry-specific" terms
  • increase the number and quality of explanations where the above is not possible
  • increase the number of links to external resources
  • provide illustrations to make things easier to understand

Introduction to the project

This page serves as a quick introduction to the project. It's written in a form of a list of answers to some of the frequently asked questions.

What is this project about?

It's about creation of user friendly environment for simulation model design and processing. It's about discovering possibilities for collaboration on certain kinds of simulation models.

At a more basic level it's about discovering a good minimal simulation architecture that's useful, extendable and easy to use.

What are the overall goals for this project?

  • provide a system for modelling and simulating real-world socio-economic systems (to try to get a better understanding of how things work and how we can influence large complex systems)
  • provide an inclusive environment for simulation modelling for anyone interested
  • provide a basic and easy to reason about simulation framework
  • provide a simple to use interface for simulation modelling, based on already existing formats
  • provide a simple interface for simulation modelling that could be used by custom applications

How useful is it right now?

Right now the project consists of a proposed system for how collaborative simulation-modelling could happen, as well as experimental software implementing things that are necessary for this to happen.

If you're ready to build from source (Rust programming language) you can already run some of the software.

See project overview for more information about the software subprojects, and the project status page to learn more about what's being actively worked on right now.

How useful could it become?

That's hard to say. It depends on how useful the base simulation engine and it's API interface is.

It's designed to be relatively basic and generic so it can scale well, but it's not certain that it will.

The design of the engine itself imposes important limitations on the possible simulations to be created for it. There are tradeoffs to be had, as with most things, and the overall design here is influenced by the larger goals of the project.

Community created content?

The goal is to create a situation where multiple users can collaborate on files organized into versioned modules.

User files (for the sake of simplicity also collectively called content) are parsed and a simulation instance is spawned using that data.

User files provide both the initial state information (here state meaning a data-based representation of an object at some point in simulation time; we call this data) as well as the computation instructions necessary for running the simulation.

Project Overview

This project is not monolithic, it's composed of a bunch of smaller subprojects.

As mentioned in the introduction, at the core of the project lies the simulation engine itself. It handles functionality related to parsing and running simulations, exposing a simple interface for interacting with the simulation data to the programmer.

Then there are the tools that provide an environment for working with the simulations. Tools also provide new layers of functionality and interoperability, for example offering a network interface (see endgame).

Finally there are games and other applications.

Simulation engine

The engine is not an executable application. It's a library that can be used by other applications to create and run simulations.

Simulation engine handles:

  • parsing input data
  • creating simulation instances
  • processing simulation instances
  • reading and writing simulation instance data

Implementation details aside, these tasks enable applications to make use of outcome simulations, including .

The engine takes care of all the details of creating and processing simulations, using the library doesn't require complete knowledge of how it works. To learn about how the engine works check out the next chapter.

Tools

To be able to run simulations we need some kind of application that makes use of the simulation engine.

The most basic tool is the command-line based endgame. Some of it's functionality is also exported as a library so it can be used within other applications. One useful example is the networking layer functionality.

Command-line is not for everyone, that's why there's also furnace, which is a GUI app with a window-based interface. It works on all popular operating systems (Linux, Windows, Mac). furnace is not exactly a replacement or alternative for endgame, rather it builds on it's features to be even more useful to the user.

Games and other applications

Anthropocene is a modern-day global strategy game. It doesn't use the simulation engine library directly, instead it uses the networking layer provided by endgame. It serves as a demonstration of how all kinds of projects, including games, can make use of outcome simulations no matter the framework or the programming language used.

Simulation Engine

Let’s dive into the simulation engine. In this chapter we’ll take a higher-level perspective, for more concrete implementation details see the code repository.

Library

Outcome simulation engine is provided as a code library. It can be used by any other program where we want to be able to work with outcome simulations.

The library exposes a set of useful functions that can be used to accomplish various tasks.

Some of the most important features of the engine are:

  • parsing user files
  • creating simulation instance
  • processing simulation instance
  • providing interface for reading and writing simulation instance data

tt

At the core of the project is an experimental simulation engine. It's designed around a few basic ideas:

  • simulation models and data are created by users, therefore
  • the engine doesn't contain simulation models or data ("moving parts" like entities and components are generic)
  • data-driven architecture, everything is based on addressed variables
  • cycle-based processing scheme, meaning one clock and a set of

Component-based

Cycle-based (sort of event-based)

Entity-component design

Outcome simulations are based on a component-based approach, where the system consists of a set of entities to which components are attached.

Entity

Entity

Additionally components contain finite-state machines. See the next sub-chapter to learn more about that.

Generic

On the level of the engine, entities and components are mostly generic - they are instead defined from user files.

Entity/Component types?

When combined with entities and components, "type" can mean different things.

Entity type

When it comes to entities, a type of an entity defines what components it gets upon instantiation.

It's a label that can be then used inside a component declaration to add. It works two ways, we can also specify components for entity types, but it's less convenient to do so.

Component type

We can use component type to specify . Declaration of a new component type can contain a .

Component type can be also used as a way of organizing components, and/or expanding the component namespace. Consider the following example:

# declare a new component type
component_type:
- id: decision
  vars:
  - id: template_var
  ...
  states:
  - id: template_state_1
  ...

# use the new component type
component:
- id: choice_213
  type: decision
  
# component `choice_213` has a var `template_var`
# /region/e_01001/decision/

Clock event

Simulation processing includes the notion of having particular clock event trigger execution of certain components that subscribe to that event.

Multiple time-scales

To reduce redundant computation we selectively set components to subscribe to events that are processed less often. This is useful for components that don't require frequent execution.

Addresses (references)

Many simulation objects can be referenced using addresses. Addresses look similar to the urls you know from your browser (or the file paths from your operating system) - multiple parts are separated by / dashes. Here are a few examples of different addresses:

# path to prop element, which points (redirects) directly to it's main var
/region/e_01001/prop/immigration_rate_daily
# explicit path to the above prop's var
/region/e_01001/prop/immigration_rate_daily/var/main
# path
/org/GER/policy/immigration_policy_1/current_state
# relative path to an element (entity is inferred from current context)
# notice lack of the dash at the front
prop/population
# dynamic path with curly brackets
/org/{prop/master_org}/prop/

Building addresses we follow this scheme:

/[entity_type]/[entity_id]/[element_type]/[element_id]/[var_type]/[var_id]

Relative address

Dash at the beginning suggests an absolute path, while no dash suggests a relative path. Relative path means that the first part of the path is omitted because it’s the same as the first part of the path of the entity where the current execution is happening (context). For example we might currently be executing instructions (commands) on some region entity m_01012 and we specify some path like “prop/population” somewhere. In such case the path will be rewritten behind the scenes to become “/region/m_01012/prop/population”.

Dynamic address

Curly braces can be used to insert some other variable’s value into the path by referencing the path to that other variable.

Synonyms

Synonym in an address is basically an item within that address that is rewritten on initial processing.

There are a few synonyms that can be used to shorten the addresses.

Examples:

reg -> region

org -> organization

prop -> property

The addresses are parsed at initialization and any recognized synonyms are treated the same as the basic (longer) variants. So for example:

# this
/org/org_id/prop/prop_id
# is the same as this
/organization/org_id/property/prop_id

Address persistence

Since the objects that can be referenced using addresses are all static, meaning they can't be added or removed after initialization, addresses themselves also don't change during the simulation run.

What can be referenced using an address

Addresses can be used to point to:

  • entities
  • elements
  • element states
  • element vars

Vars (variables)

Var is a piece of data which is accessible through a reference (based on idea of variable from computer science). This means each var is composed of two parts:

  • reference (in our case it's an address)
  • value (data, could be any of the types defined below, like a number or a string of characters)

Vars exist in the context of elements. Since the elements are static objects within the simulation vars are also static - they persist no matter what since simulation instance initialization.

Vars are mutable, meaning their value can be changed during simulation run. At the same time references to vars (their addresses) cannot be changed after initialization (persistence).

How are vars implemented?

Vars are key-value pairs that are persistent throughout the simulation. They are stored inside the Database object, which itself is a key-value store (listing).

Vars are declared and used within the context of elements and their system of states with sets of instructions (commands).

Address of a var is it's key, and the piece of data is it's value (whether a number, a string or other kind of data).

During initialization the vars as they appear in the module files are transformed into pointers to actual data stored in the Database that the program can understand. The exception here are vars referenced using dynamic addresses - these take longer time to retrieve because the program has to look up the database first (if it were a usual static reference it would already have the reference to the desired data).

# example of an element declaration with a var declaration included
element:
  - id: forest_coverage
    # lack of type specification means it's a generic element
    entity: region
    vars:
      # address for the following var could be
      # /reg/e01001/law/some_law/num/some_num_var
      - name: some_num_var
        type: number
        default: 5

Declaration and persistence

Vars need to be declared within an element declaration under vars entry.

Vars need to be registered at initialization, they can't be added after initialization.

info Remember: Var registration is a process of registering a var for an entity type, instead of for a specific entity. For example when registering a new var for the region level each region entity will receive an instance of this new var and will be able to use it.

Var types

There are several types of vars (based on different kinds of values they hold):

  • number
  • string
  • bool
  • number table
  • string table
  • bool table

Database object

Database is the key-value store for vars. It's an object relevant in the context of a single simulation run, as it's created on initialization and discarded after the simulation run is finished. At any point of it's existence after the initialization, the information inside the database object can be exported to a snapshot, allowing for later reconstruction.


showToc: true

Commands

Commands (cmds) are the micro-programs that are invoked by the element states. There are simple commands (cmd) and complex commands (ccmd). Complex commands are parsed and cut into multiple simpler commands and stored this way for later processing.

Commands are hardcoded (though there is also user_cmd cmd allowing for user defined commands??). Commands can do many different things and take varying numbers of arguments.

??Commands return some value (base value class, so either string, num or bool) as their return value.

Let's look at all the available cmds.

Simple

set

set [var_address_1] [var_address_2]

Sets the value at var_address_1 to the value of var_address_2.

log

log [log_message]

Logs a message to the default log store.

Complex

oper

oper [var_address_1] [operation_sign] [(calc)]

Operates on the var_address_1, converts contents of the third argument into separate calc

Interface

Engine library provides an interface for:

  • creating sim instance from data
  • creating snapshot from sim instance
  • reading data from sim instance
  • writing data to sim instance

Data Management

Data management is about handling, organising and distributing the data that is used to create and run simulations. The main features relevant here are mods, scenarios and snapshots.

In one sentence: Mods are collections of instructions, scenarios are collections of mods, snapshots are saved states of simulation instances (like a game save).

Let’s go through those three levels of data and see how they are related to each other.

Module

Modules, mods for short, are, simply put, collections of data that can be gathered together and used to spawn a simulation instance. It’s helpful to think of mods as packages - mods allow for modularity, in the sense that we can put different collections of mods together and achieve a working simulation model that can be run.

This ability to combine multiple mods is the key here. For managing multiple mods within one “environment” we’re moving into the domain of scenarios.

Mod within the larger file structure

Mods always exist in dedicated mods directory. This directory can be found inside larger structures of scenarios, which almost by definition are collections of multiple mods.

File structure within the module

There are not many strict requirements in terms of internal organization of files in a module. One such requirement is the module.yaml file.

module.yaml

Each mod needs a module.yaml file present in it's top directory.

Required:

  • name unique name of the module, string without spaces
  • version version of the module
  • outcome version of the outcome architecture the module is written for

Optional:

  • title evan more human readable version of the name
  • description briefly about the mod
  • description_long longer description
  • author group or person behind the mod
  • website place to get more information about the mod and people behind it
  • dependencies list of required modules for this module to work, see more about dependencies
# example of a module.yaml file
name: test_mod
title: Test mod
description: Just testing.
description_long: This is just a testing module, not really usable yet.
author: John Doe
website: example.com
version: 0.0.1
outcome: 0.0.1
dependencies:
  base: 0.0.1

Flexibility within the mod

Apart from the required files, there is much flexibility when it comes to organization of non-essential files (user-written module files) inside the mod. This flexibility is possible because the files themselves specify everything inside them - the declarations made within module files don't need any additional context. Thus, structure of directories within the mod, even the names of the files are not an essential part of the module file processing.

Program reading a module will read all files (given they have the proper yaml/json extensions).

Organization of files into directories can be useful, so can be certain approaches to naming the module files.

Scenario

Scenario wraps a collection of mods into one simulation environment, so to speak. Simulation instance is always spawned using a single scenario as input. This is also true for initiating simulation instance using a snapshot - in that case the snapshot points to a scenario to be used.

File structure

asd

scenario.yaml

Snapshot

Snapshot holds initial-state data which can be used to load specific state for a simulation instance. Snapshot still needs to point to a scenario so a simulation instance can be spawned. Snapshot only provides data which can be loaded onto an already existing simulation.

Snapshot must contain:

  • serialized copy of the database object
  • snapshot metadata

tip TODO: Snapshot could also optionally contain a collection of archived states for past simulation ticks.

Keep in mind that snapshot data is loaded only after the simulation instance has been spawned. This means that if there is any data on the snapshot that corresponds to an item that doesn’t exist in the initialized simulation instance then it will not be loaded. Also the other way around, if the snapshot doesn’t contain data for something that is defined and exists at the simulation instance, then the default value from the declaration will be used.

Dependencies

Execution context considerations

Locality focus

Static vs dynamic references

Non-essential data

i18n

Illustrations