Step 8: Statistics calculation
We have created a simulation model that gets initialized by a scenario, generates random events (transportation requests) and assigns them to trucks. The only output that we get from the terminated model is its finish time - it gets printed to the console.
We will now calculate the useful experiment statistics.
Service level
Service level is the fraction of requests that have been fulfilled in time. There are four cases here:
-
FULFILLED_IN_TIME: a request is completed, its completion time is earlier than the deadline time.
-
BELATED: a request is completed, its completion time is after the deadline time.
-
WILL_BE_BELATED: a request is not completed, its deadline is in the past.
-
INDEFINITE: a request is not completed, its deadline is in the future.
We use the following formula for the service level:
SL = (# of FULFILLED_IN_TIME) / (# of COMPLETED + # of WILL_BE_BELATED),
where # of COMPLETED = # of FULFILLED_IN_TIME + # of BELATED
.
Note that INDEFINITE requests are ignored when calculating the service level, since we cannot know in advance whether this request will be fulfilled in time.
Expenses
Expenses will be calculated as the total costs over all trucks.
If we divide the expenses by the service level (in percents), we get the expenses per % of service level - that’s the value we want to minimize.
The Statistics class
Add the following code to the new Statistics
class in the 'tutorial.model' package:
package tutorial.model;
import com.amalgamasimulation.utils.Utils;
public class Statistics {
private final Model model;
public Statistics(Model model) {
this.model = model;
}
public double getExpenses() {
return model.getTrucks().stream().mapToDouble(Truck::getExpenses).sum();
}
public double getServiceLevel() {
int fulfilledInTimeRequests = 0;
int completedOrWillBeBelatedRequests = 0;
for (TransportationRequest request : model.getRequests()) {
if (request.isCompleted() && request.getCompletedTime() <= request.getDeadlineTime()) {
fulfilledInTimeRequests++;
}
if (request.isCompleted() || request.getDeadlineTime() <= model.engine().time()) {
completedOrWillBeBelatedRequests++;
}
}
return Utils.zidz(fulfilledInTimeRequests, completedOrWillBeBelatedRequests);
}
public double getExpensesPerServiceLevelPercent() {
return Utils.zidz(getExpenses(), getServiceLevel() * 100);
}
}
Changes to the Model class
The Statistics
object is created in the Model
class constructor.
We also need to store this object and return it from the Model
to be able to extract the
experiment output data from our model.
So, we add a new field, initialize it in the Model
class constructor, and return it in a new getter method.
Add a new field to the Model
class:
private final Statistics statistics;
Add the Statistics
initialization code to the Model
class constructor:
statistics = new Statistics(this);
Finally, add a new getter method:
public Statistics getStatistics() {
return statistics;
}
The Model
class is now ready to report the simulation results.
Changes to the ExperimentRun class
In our Main.main()
method, we do not work with the Model
object directly.
Instead, we create an ExperimentRun
object.
We will need the ExperimentRun
object to report the results of the simulation.
Add a new import statement in the 'ExperimentRun.java' file:
import tutorial.model.Statistics;
Add the following new method to the ExperimentRun
class:
public Statistics getStatistics() {
return model.getStatistics();
}
Changes to the Main class
Now we are ready to update the Main.runExperiment()
method to print the experiment statistics.
Add new import statements to the 'Main.java' file:
import tutorial.model.Statistics;
Add the Main.runExperimentWithStats()
method:
private static boolean headerPrinted = false;
private static void runExperimentWithStats(Scenario scenario, String scenarioName) {
ExperimentRun experiment = new ExperimentRun(scenario, new Engine());
experiment.run();
Statistics statistics = experiment.getStatistics();
if (!headerPrinted) {
System.out.println("Scenario \tTrucks count\tSL\tExpenses\tExpenses/SL");
headerPrinted = true;
}
System.out.println("%-20s\t%12s\t%s\t%s\t%s".formatted(scenarioName, scenario.getTruckCount(),
Formats.getDefaultFormats().percentTwoDecimals(statistics.getServiceLevel()),
Formats.getDefaultFormats().dollarTwoDecimals(statistics.getExpenses()),
Formats.getDefaultFormats().dollarTwoDecimals(statistics.getExpensesPerServiceLevelPercent())));
}
Add the new createAndRunOneExperimentWithStats()
method:
private static void createAndRunOneExperimentWithStats() {
Scenario scenario = new Scenario( 1,
TRUCK_SPEED,
INTERVAL_BETWEEN_REQUESTS_HRS,
MAX_DELIVERY_TIME_HRS,
warehouses,
stores,
routeLengthContainer,
LocalDateTime.of(2023, 1, 1, 0, 0),
LocalDateTime.of(2023, 1, 1, 12, 0));
runExperimentWithStats(scenario, "scenario");
}
Replace the Main.main()
method:
public static void main(String[] args) {
createAndRunOneExperimentWithStats();
}
Check the result
Run the program:
0,000 Request #1 created 0,000 Task #1 : TRANSPORTATION_STARTED. Request #1; Truck #1 at Store 1; From Warehouse 3 -> To Store 2 1,024 Request #2 created 2,500 Task #1 : TRANSPORTATION_FINISHED 2,500 Task #2 : TRANSPORTATION_STARTED. Request #2; Truck #1 at Store 2; From Warehouse 1 -> To Store 3 3,733 Request #3 created 3,871 Request #4 created 3,877 Request #5 created 4,037 Request #6 created 4,750 Task #2 : TRANSPORTATION_FINISHED 4,750 Task #3 : TRANSPORTATION_STARTED. Request #3; Truck #1 at Store 3; From Warehouse 3 -> To Store 3 4,789 Request #7 created 6,750 Task #3 : TRANSPORTATION_FINISHED 6,750 Task #4 : TRANSPORTATION_STARTED. Request #4; Truck #1 at Store 3; From Warehouse 2 -> To Store 1 7,358 Request #8 created 8,120 Request #9 created 8,197 Request #10 created 9,000 Task #4 : TRANSPORTATION_FINISHED 9,000 Task #5 : TRANSPORTATION_STARTED. Request #5; Truck #1 at Store 1; From Warehouse 1 -> To Store 3 9,129 Request #11 created 10,034 Request #12 created 10,803 Request #13 created 11,250 Task #5 : TRANSPORTATION_FINISHED 11,250 Task #6 : TRANSPORTATION_STARTED. Request #6; Truck #1 at Store 3; From Warehouse 1 -> To Store 2 Scenario Trucks count SL Expenses Expenses/SL scenario 1 57,14% $ 420,00 $ 7,35
You’ll notice a new line in the end that shows the overall statistics. Let’s check if it is correct.
Of 13 requests created:
-
Requests #1, #2, #3, and #4 are FULFILLED_IN_TIME (i.e., within 6 hours).
-
Request #5 is BELATED.
-
Request #6 has a task that has started, but it WILL_BE_BELATED.
-
Request #7 has no task yet, and it also WILL_BE_BELATED, since it was created earlier than 6 hours (max delivery time) before the simulation has finished.
-
Requests #8 up to #13 have INDEFINITE state.
So, there are 4 requests FULFILLED_IN_TIME, 1 BELATED request, and 2 WILL_BE_BELATED requests.
According to the SL formula presented earlier, we get:
4 / (4 + 1 + 2) = 57.14%
.