Performance in Open-ended Dynamic Tasks
DSF: The Dynamic Stock & Flows Task
DSF is a generic dynamic decision making task that involves the regulation of a stock (maintain a quantity at a target level) through inflow (increase the stock) and outflow (decrease the stock) rates. DSF was designed to study and explain human dynamic decision making processes. The full capabilities of DSF are described elsewhere (Dutt & Gonzalez, 2007; Gonzalez & Dutt, 2007), therefore we will only give a brief overview of the DSF capabilities relevant for the empirical data and modeling for the purpose of this challenge.
The goal in DSF is to reach and maintain the level of water in a tank at a target level over a number of time periods. The level of water in the tank is the stock that increases with the inflows and decreases with the outflows. There are two types of inflows and outflows in this task: those that are exogenous (outside of the decision maker’s control) and those endogenous (under the decision maker’s control). The exogenous flows are called Environmental Inflow (that increases the level of the stock without the user’s control) and the Environmental Outflow (that decreases the level of stock without user’s control). The endogenous flows are User’s Inflow and Outflow. Setting the UserInFlow and OutFlow are the main decisions made by the user in each time period. In addition, if the system has a delay, defined by number of time periods, the stock level will also depend on the Supply Line (SL) (Sterman, 1989). The SL works as a First-in-First-out (FIFO) queue and contains the User Inflow and/or User Outflow values that have been placed but not yet received in the system.
The stock level at time t can be defined as follows:
Stock (t) = Stock (t-1) + [Environment Inflow (t-1) + User Inflow(t-1)] - [Environment Outflow (t-1) + User Outflow(t-1)] + SL(t-1-delay)
The video below presents the graphical user interface of DSF. At each time period users see the values of Environment Inflow and Outflow, values of User Inflow and Outflow, the amount of water in the tank (stock) and the goal level. At each time period, users submit two values, one for user inflow and another for user outflow. These inputs can take any positive value, including zero. Users may also receive a ‘bonus’ performance monetary incentive in each time period in which they were close enough to the target level.
Despite its seeming simplicity, controlling the DSF is a slow process for most subjects (Gonzalez & Dutt, 2007; Dutt & Gonzalez, 2007). For example, Cronin, Gonzalez & Sterman (2009) found that in a sample of highly educated graduate students with extensive technical training nearly half were unable to predict the qualitative path of a stock given very simple patterns for its inflow and outflow. Subject learning was slow and ultimately sub-optimal even in the simple conditions of the task, for example, that of controlling the system due to an increasing inflow and zero outflow (Gonzalez & Dutt, 2007; Dutt & Gonzalez, 2007). Moreover, Cronin and Gonzalez (2007) presented subjects with a series of manipulations related to the form of information display, context of the task, incentives and others factors intended to help the subject understand the task, demonstrating that the difficulty in understanding stocks and flows is not due to lack of information or the form of information presentation.
The DSF task environment requires a Windows platform and can be run in two modes. First, the DSF can be run as a live experiment so that a human can interact with the same task environment used in human experiments. In this way, modelers can gain hands-on experience with the task and use this experience to inform the development of their own models. Second, the DSF environment can be run as a constructive simulation, without the user interface, in a faster-than-real time mode with the participants' computational models interacting directly with the task environment.
While the task environment only runs on Windows OS, participants are free to run their clients on Macs or Linux machines as well as Windows, since the DSF task environment interacts with the cognitive model through a text-based TCP/IP socket communication.
Click here to download the user interface version of DSF. (Last updated 4/3/09)
Click here to download the TCP/IP socket version of DSF. (Last updated 4/27/09)
See the DSF documentation below for more information on running and uninstalling DSF.
Instructions for running the user interface version of DSF:
Instructions for running TCP/IP socket version of DSF:
2. Unzip the contents of the zip file (this will give 2 files and a folder). The two files are java client files for running/testing the simulation (both the .java and compiled .class files are included so that one DOES NOT need to download the java SDK: simply go to the prompt locate the class file on the command prompt and run the class file by writing > java DSFClient (enter)). In case you want to change the IP address etc. for the sample java socket program, you might need to compile the .java file once again by > javac DSFClient.java.
3. Double click the “setup.exe” file inside the unzipped folder
4. Simply follow the installation instructions (if you don’t want to change anything simply do Next, Next…. until you are finished with the install)
5. After install completes the Dynamic Stocks And Flow program will be installed in Start -> Programs -> Dynamic Stocks And Flow folder (go to this location and click the created executable file which will start DSF with a welcome screen)
Structure of SETUP.TXT File as part TCP/IP socket version of DSF:
Socket protocol documentation
The Dynamic Stocks and Flows (DSF) simulation uses a text-based TCP/IP socket protocol to communicate with any other software package. On double clicking the executable file, a "Welcome" screen is presented and at this time a server socket is opened on IP address of "localhost" (i.e. 127.0.0.1) using port number 9548. Software packages that need to connect to DSF as "clients" need to open a client port after the "Welcome" screen is presented in DSF.
When you see the welcome screen come up, go to command prompt of your computer (on Windows XP/Vista click start ->Run... -> type "cmd" without quotes). Navigate to the folder where the Java socket client file is stored (DSFClient.java file; using cd <path> command). After this on the command prompt, compile the java file and get its executable (thus, type: javac <filename>.java on command prompt or simply use: java DSFClient which does not need java SDK). Now run the compiled file by typing: java <filename>. This step should give you a success for the socket connection.
Remember that for any socket client to connect to DSF, the welcome screen should be present prior to running the socket program. After the last step, type the first login command and press enter:
ENTRY-ID Varun1 VERSION BATCH/NONBATCH ACTR/NONACTR
(Varun1 is any dummy name and if BATCH is entered then the simulation runs faster and does not show the DSF interface where as when NONBATCH is entered the simulation shows the DSF interface and runs slower; where the last keyword depends upon programming language used for the client). Remember the instruction set for communication on the socket is case sensitive so enter it exactly as it appears here (except for dummies occurring in the instruction text like Varun1). The simulation should start up and you should receive the first message of the state of the simulation. The STATE is just an identifier of the message string and has no purpose. The value following the TIME tells the time period of the simulation. The value following GOAL suggests the goal amount, which is set as 4.000. The value following AMOUNT is the current amount in the DSF tank.
Similarly, values following EI, EO are the values of environment inflow, environment outflow displayed on the GUI for the current time period TIME. Also UI and UO are the values of user inflow and user outflow that the subject entered in the last time period. As you’ll see for the 0th TIME the EI, EO, UI and UO values are -1 or unknown. Now type a decision.
For example, DECISION UI 2 UO 4 EI 3 EO 2 (notice that we send not only the UI and UO but also EI and EO from the model) and press enter to send this to the simulation. The simulation would send you the next state after executing the decision. Once a model of the task has been developed, the model should determine the UI and UO values, while the EI and EO values can be set by a program to implement any environmental input/output function.
Further, you can send SIMULATION RESET anytime to DSF via socket connection, which will cause the simulation to return back to the welcome screen (and end the current participant/model run). The SIMULATION RESET can either be followed by the new login string like:
ENTRY-ID Varun2 VERSION BATCH/NONBATCH
(to start a new participants) or QUIT command (to exit DSF). Once the end time period is reached, the simulation would send you END_PROGRAM when it gets a decision string like:
DECISION UI <value>UO <value> EI <value> EO <value>.
At this time send: QUIT (to close the socket and DSF) or SIMULATION RESET followed by login instruction ENTRY-ID Varun3 VERSION BATCH/NONBATCH (to run a new participant). The DSF simulation sends an acknowledgement “SIMULATION_RESETTED” after resetting itself. The login instruction with ENTRY-ID should be given after receiving the SIMULATION_RESETTED acknowledgement from DSF.