Step 8: Statistics calculation

We have created a simulation model that gets initialized by a scenario, generates random events (transportation requests) and assigns them to trucks. The only output that we get from the terminated model is its finish time - it gets printed to the console.

We will now calculate the useful experiment statistics.

Service level

Service level is the fraction of requests that have been fulfilled in time. There are four cases here:

  1. FULFILLED_IN_TIME: a request is completed, its completion time is earlier than the deadline time.

  2. BELATED: a request is completed, its completion time is after the deadline time.

  3. WILL_BE_BELATED: a request is not completed, its deadline is in the past.

  4. INDEFINITE: a request is not completed, its deadline is in the future.

We use the following formula for the service level:

SL = (# of FULFILLED_IN_TIME) / (# of COMPLETED + # of WILL_BE_BELATED),

where # of COMPLETED = # of FULFILLED_IN_TIME + # of BELATED.

Note that INDEFINITE requests are ignored when calculating the service level, since we cannot know in advance whether this request will be fulfilled in time.

Expenses

Expenses will be calculated as the total costs over all trucks.

If we divide the expenses by the service level (in percents), we get the expenses per % of service level - that’s the value we want to minimize.

The Statistics class

Add the following code to the new Statistics class in the 'tutorial.model' package:

Statistics.java
package tutorial.model;

import com.amalgamasimulation.utils.Utils;

public class Statistics {
    private final Model model;
    
    public Statistics(Model model) {
        this.model = model;
    }
    
    public double getExpenses() {
        return model.getTrucks().stream().mapToDouble(Truck::getExpenses).sum();
    }
    
    public double getServiceLevel() {
        int fulfilledInTimeRequests = 0;
        int completedOrWillBeBelatedRequests = 0;
        for (TransportationRequest request : model.getRequests()) {
            if (request.isCompleted() && request.getCompletedTime() <= request.getDeadlineTime()) {
                fulfilledInTimeRequests++;
            }
            if (request.isCompleted() || request.getDeadlineTime() <= model.engine().time()) {
                completedOrWillBeBelatedRequests++;
            }
        }
        return Utils.zidz(fulfilledInTimeRequests, completedOrWillBeBelatedRequests);
    }
    
    public double getExpensesPerServiceLevelPercent() {
        return Utils.zidz(getExpenses(), getServiceLevel() * 100);
    }
}

Changes to the Model class

The Statistics object is created in the Model class constructor. We also need to store this object and return it from the Model to be able to extract the experiment output data from our model.

So, we add a new field, initialize it in the Model class constructor, and return it in a new getter method.

Add a new field to the Model class:

Model.java, new Statistics field
private final Statistics statistics;

Add the Statistics initialization code to the Model class constructor:

Model.java, Statistics initialization
statistics = new Statistics(this);

Finally, add a new getter method:

Model.java, Statistics getter
public Statistics getStatistics() {
    return statistics;
}

The Model class is now ready to report the simulation results.

Changes to the ExperimentRun class

In our Main.main() method, we do not work with the Model object directly. Instead, we create an ExperimentRun object. We will need the ExperimentRun object to report the results of the simulation.

Add a new import statement in the 'ExperimentRun.java' file:

import tutorial.model.Statistics;

Add the following new method to the ExperimentRun class:

ExperimentRun.java, Statistics getter
public Statistics getStatistics() {
    return model.getStatistics();
}

Changes to the Main class

Now we are ready to update the Main.runExperiment() method to print the experiment statistics.

Add new import statements to the 'Main.java' file:

import tutorial.model.Statistics;

Add the Main.runExperimentWithStats() method:

Main.java, new runExperimentWithStats() method
private static boolean headerPrinted = false;
private static void runExperimentWithStats(Scenario scenario, String scenarioName) {
    ExperimentRun experiment = new ExperimentRun(scenario, new Engine());
    experiment.run();
    Statistics statistics = experiment.getStatistics();
    if (!headerPrinted) {
        System.out.println("Scenario            \tTrucks count\tSL\tExpenses\tExpenses/SL");
        headerPrinted = true;
    }
    System.out.println("%-20s\t%12s\t%s\t%s\t%s".formatted(scenarioName, scenario.getTruckCount(),
            Formats.getDefaultFormats().percentTwoDecimals(statistics.getServiceLevel()),
            Formats.getDefaultFormats().dollarTwoDecimals(statistics.getExpenses()),
            Formats.getDefaultFormats().dollarTwoDecimals(statistics.getExpensesPerServiceLevelPercent())));
}

Add the new createAndRunOneExperimentWithStats() method:

Main.java, new createAndRunOneExperimentWithStats() method
private static void createAndRunOneExperimentWithStats() {
    Scenario scenario = new Scenario(   1,
                                        TRUCK_SPEED,
                                        INTERVAL_BETWEEN_REQUESTS_HRS,
                                        MAX_DELIVERY_TIME_HRS,
                                        warehouses,
                                        stores,
                                        routeLengthContainer,
                                        LocalDateTime.of(2023, 1, 1, 0, 0), 
                                        LocalDateTime.of(2023, 1, 1, 12, 0));
    runExperimentWithStats(scenario, "scenario");
}

Replace the Main.main() method:

public static void main(String[] args) {
    createAndRunOneExperimentWithStats();
}

Check the result

Run the program:

0,000	Request #1 created
0,000	Task #1 : TRANSPORTATION_STARTED. Request #1; Truck #1 at Store 1; From Warehouse 3 -> To Store 2
1,024	Request #2 created
2,500	Task #1 : TRANSPORTATION_FINISHED
2,500	Task #2 : TRANSPORTATION_STARTED. Request #2; Truck #1 at Store 2; From Warehouse 1 -> To Store 3
3,733	Request #3 created
3,871	Request #4 created
3,877	Request #5 created
4,037	Request #6 created
4,750	Task #2 : TRANSPORTATION_FINISHED
4,750	Task #3 : TRANSPORTATION_STARTED. Request #3; Truck #1 at Store 3; From Warehouse 3 -> To Store 3
4,789	Request #7 created
6,750	Task #3 : TRANSPORTATION_FINISHED
6,750	Task #4 : TRANSPORTATION_STARTED. Request #4; Truck #1 at Store 3; From Warehouse 2 -> To Store 1
7,358	Request #8 created
8,120	Request #9 created
8,197	Request #10 created
9,000	Task #4 : TRANSPORTATION_FINISHED
9,000	Task #5 : TRANSPORTATION_STARTED. Request #5; Truck #1 at Store 1; From Warehouse 1 -> To Store 3
9,129	Request #11 created
10,034	Request #12 created
10,803	Request #13 created
11,250	Task #5 : TRANSPORTATION_FINISHED
11,250	Task #6 : TRANSPORTATION_STARTED. Request #6; Truck #1 at Store 3; From Warehouse 1 -> To Store 2
Scenario                Trucks count    SL      Expenses        Expenses/SL
scenario                           1    57,14%  $ 420,00        $ 7,35

You’ll notice a new line in the end that shows the overall statistics. Let’s check if it is correct.

Of 13 requests created:

  1. Requests #1, #2, #3, and #4 are FULFILLED_IN_TIME (i.e., within 6 hours).

  2. Request #5 is BELATED.

  3. Request #6 has a task that has started, but it WILL_BE_BELATED.

  4. Request #7 has no task yet, and it also WILL_BE_BELATED, since it was created earlier than 6 hours (max delivery time) before the simulation has finished.

  5. Requests #8 up to #13 have INDEFINITE state.

So, there are 4 requests FULFILLED_IN_TIME, 1 BELATED request, and 2 WILL_BE_BELATED requests.

According to the SL formula presented earlier, we get: 4 / (4 + 1 + 2) = 57.14%.