A comparison between InfoWorks WS Pro and simplified demand prediction models

This topic shows how much better a predictor of demand InfoWorks WS Pro is when results are compared with those of simplified models of the type demand today at 3pm is the same as demand yesterday at the same time or use data from the week before. InfoWorks WS Pro parameters explain more of the total variation than parameters used in the simplified models (better coefficient of determination) and the additional components used in InfoWorks WS Pro are statistically significant as shown by the F-test that has been carried out on the data (see below).

InfoWorks WS Pro has the following added benefits:

• InfoWorks WS Pro provides a good fit between observed and predicted values even if telemetry data is not available.
• Temperature data can be used to further increase predictive power.
• InfoWorks WS Pro predictions are fully automated.
• InfoWorks WS Pro has sophisticated connectivity to many types of databases (e.g. Oracle, SQL Server, ODBC, PI, ClearSCADA, iHistorian, JET...).

The question is whether InfoWorks WS Pro has a better predictive power than the use of yesterday’s demand for predicting today’s demand. For example, if we wanted to predict the demand at 3pm today, why not just take yesterday’s measured demand at 3pm instead of using InfoWorks WS Pro?

To test this, let us use InfoWorks WS Pro to build this simple prediction model by setting all the InfoWorks WS Pro parameters to zero with the exception of PSI24, which should be set to 1 in the Model Parameters page of the Properties dialog.

This way, only the measured demand from 24 hours ago will be used to predict the current demand (See Figure 1).

Figure 1

InfoWorks WS Pro can then be used to verify the model using real data.

When using the full InfoWorks WS Pro model, the coefficient of determination (R²) is 0.969.

When using the simplified model, the coefficient of determination is 0.828. Figure 2 shows a graph of predicted versus measured demand for the full InfoWorks WS Pro model. This graph is visible in the Forecast Run page of the Properties dialog and the forecast may be tested for a time other than the training period (Time Range section).

Figure 2

Figure 3 shows the equivalent graph for the simplified model:

Figure 3

By visually comparing both graphs it can be seen that the fit is better for the InfoWorks WS Pro model.

A statistical analysis performed on the data shows that the additional InfoWorks WS Pro parameters (going from the simplified to the full model) are statistically significant (see Table 1).

Table 1: Results for the full InfoWorks WS Pro model.

When performing an incremental F-test for additional model parameters, the F value is given by the following equation:

where:

is the unconstrained model R² value

is the constrained model R² value

J is the number of additional parameters

N is the number of samples

K is the total number of parameters in the unconstrained model

The unconstrained model is the model with the additional parameters, the constrained model is the model without the additional parameters.

Another comparison could be performed between InfoWorks WS Pro and the data from exactly one week ago. In this case, the coefficient of determination (R²) is 0.755, which is slightly worse than the value for the simplified daily model (0.828). As this simplified weekly model cannot be directly run in InfoWorks WS Pro the data has been exported to CSV and the calculations have been performed in Microsoft Excel.

Figure 4 shows the results for the simplified weekly model.

Figure 4

Simplified models are strong predictors. They explain upward of 75% of the total variation. However InfoWorks WS Pro is significantly better as 97% if the variation is explained. The standard error of InfoWorks WS Pro predictions is about three times smaller than for simpler models. This demonstrates the value of InfoWorks WS Pro, particularly given the fact that the process is automated and good predictions are achieved even when telemetry fails.