Prediction of S&P 500 Index Using HAR-RV Models with Structural Breaks, Day-of-the-Week Effect and Trading Volume

. Economic trends are one the crucial research topics. Standard and Poor's 500 index (S&P500) is an indicator of economic change with diversification. In this article, the time series models of forecasting realized volatility, regarding the heterogeneous autoregressive theory, are evaluated. For the day, week, and month predictions, heterogeneous autoregressive models with realized volatility (HAR-RV) type models take structural breaks, the day-of-the-week impact, and trade volume into account. Structure breaks and day-of-the-week effects have positive effects on in-sample prediction, while there is not enough evidence to show that trading volume is significant for in-sample prediction. For out-sample prediction, simple loss functions and the Diebold Mariano test are employed to compare the capability of HAR-RV type models in this article for out-sample performance. The HAR-RV, HAR-RV-VOL, and HAR-RV-WV models are regarded to be accurate for short-and mid-term out-sample prediction. The HAR-RV, HAR-RV-SV, and HAR-RV-SWV models are thought to be significant over the long run.


Introduction
Standard and Poor's 500 Index, a capitalization-weighted indicator, monitors the stock performance of 500 large-cap stocks listed on American exchanges.It gauges the performance of the USA economy by tracking market value fluctuations of these equities that represent the leading industries, where stocks included are selected using eight main criteria to reflect the value of comparable businesses [1].Risk is more varied and can reflect a variety of market developments when there are many enterprises and a wide range of sectors.The diversification of the S&P 500 enables traders to monitor the performance of both the index itself and companies in the same industries.With these characteristics, S&P 500 is chosen in this article.
Realized Volatility (RV) is the assessment of variation in returns for an investment product, based on its historical returns over a specified period.RV, with the advantage of the accuracy of the volatility in prediction, contains more information about the market than traditional volatility based on low-frequency models, for example, GARCH.
Structural break refers to the abruptly altering point in a time series at a certain moment in time.Time series data always include significant inaccuracy, and structural flaws can lead to significant forecasting errors and overall model unreliability.Time series data also usually have ambiguous patterns or features.Understanding the real mechanisms behind collected data may help us comprehend, forecast, and make decisions.This is made possible by spotting structural flaws in models.
The day-of-the-week effect refers to the stock market's yield fluctuation within a week.It describes the propensity of stocks to deliver comparatively high returns on a particular day, compared with other days in the week.For Standard & Poor's 500 indexes, a rolling sample test together with the GARCH model is employed, showing that Tuesday anomalies are prominent [2].
In-depth studies have been focused on RV-type models, modeling realized volatility using various patterns to increase its accuracy.For the first time, concerns with forecasting were studied using highfrequency data-based RV as a surrogate measure of volatility [3].In 2009, Corsi [4] raised the HAR-RV model, demonstrating that it is more capable of predicting volatility than the GARCH model.Based on the HAR-RV model, the HAR-CJ model is built in 2003 [5], together with the HAR-RV model with jump variance.In these two models, RV is divided into continuous sample path variance and jump variance.Later, the HAR-RV model considering asymmetry and memory over the long run was proposed in 2012, which provides evidence that long-term memory contains information on volatility predictions [6].The purpose of the HAR-CJN model, which combines the HAR model with continuous sample route variance, jump variance, and overnight return variation, was to examine the importance of overnight information [7].
In this article, the HAR-RV models are employed to forecast realized volatility.Throughout the forecasting process, models would be built to examine the factor's impacts on the Standard & Poor's 500 Index prediction accuracy.Factors examined in this paper include structural fractures, day-ofthe-week effect, and trading volume.To locate the structural fractures, the Iterated Cumulative Sums of Squares Algorithm is utilized which finds 17 structural breaks with a threshold of 1.628.There is a dispute over the importance of structural breaks for in-sample prediction.Depending on the p-value, certain structural fractures are regarded as precisely important, while others are not.For mid-and long-term predictions, Thursday is important, and Wednesday and Friday have an impact on 1-day predictions.The fact that the day-of-the-week effect provides a negative influence on forecasting the S&P 500 is supported by the reality that most of its coefficients are negative.Adj. 2 is used to determine the in-sample test's accuracy.In contrast to the other two, which both improve prediction accuracy, trading volume scarcely changes.For out-sample prediction, MAE, MSE and MAPE, and the Diebold Mariano test (DM test) are used to evaluate and there is not enough evidence to show that dummy variables should not be ignored in out-sample prediction.The DM test's results show that the HAR-RV, HAR-RV-VOL, and HAR-RV-WV models are significant for 1-day and 1-week prediction, but the DM test's p-value indicates that it is impossible to detect the accuracy relationship between any of them.The accuracy correlations between the HAR-RV, HAR-RV-SV, and HAR-RV-SWV models, each of which owns accuracy and significance, are underdetermined for 1-month prediction.
There are 4 sections to the paper.A brief summary of this work is provided in the Session 1 Introduction as stated above.The methodology session is covered in Section 2, along with data processing for realized volatility, dummy variables of three factors about structure, weekday, and volume per day, and HAR-type models that take the aforementioned elements into account.The result analysis session for the in-and out-sample is in Section 3.

Data
Based on the closure price of the 5-minute high-frequency data set, realized volatility data is calculated.The Iterated Cumulative Sums of Squares Algorithm (ICSS Algorithm) is used to determine the structural breaks of the S&P 500.Dummy variables are used to calculate the impact factors mentioned before.The conclusions are provided in the last section.For all the tables in this paper, the number in parentheses is t statistics and * means p-value < 0.1, ** means p-value < 0.05 and when p-value < 0.01, one more star is added.

Realized volatility
The financial markets' ability to forecast volatility is significantly influenced by sample frequency.Microscopic noise will result from an excessive sample frequency.If the frequency is low, there will be some information loss about the daily volatility.Based on the previous research [8,9], In this paper, data for a 5-minute frequency is selected to measure RV of S&P 500, which comes from Dukascopy Swiss Banking Group from 1st January 2017 to 21st August 2022, 592704 items of raw data.Based on the definition of RV, 1453 daily realized volatility data is available after eliminating the vacancy value from 31st January 2017 to 19th August 2022.
For daily RV of HAR-RV model on the  ℎ trading day, it is denoted by  , .The expression of it is as: where  , is return rate in the  ℎ 5-minute trading period of the  ℎ trading day.
, = ( , −  ,−1 ) × 100. ( Because the original return is always too small for easy observation and measurement, which is a logarithmic difference between the close of the  − 1 ℎ 5-minute trading period and the  ℎ 's.For easy observation, the logarithmic rate of return is multiplied by a hundred as shown in the following formula.As there are five trading days per week and the number of the annual trading day is around 258.Based on the trading period of the S&P 500, the weekly and monthly RV  , and  , for the  ℎ trading day can be expressed by the formula below.For  , , D is 5.For  , , D is 21. (3) Fig. 1 S&P 500 Daily RV from 31st January 2017 to 19th August 2022 Fig. 1 plots the daily realized volatility, from which structural breaks can be found between 2017 and 2022.Major alterations in economic conditions, such as shifts in policy, calamities, and wars, are referred to as structural breaks.These frequently alter the specifications of economic variables.In this paper, structural breaks would be considered while predicting the realized volatility.

Structural breaks
To identify the structural breaks, ICSS Algorithm is employed.Suppose there are T sample observations.Based on the process of employing the ICSS algorithm [10], the cumulative sum of squares (CSS) for the residuals of all possible structural break observations from 1 to k is defined as where k = 1, 2, ..., T and   represent the CSS residuals covering the entire sample period.  2 is the unconditional variance for the  ℎ interval, where i =1, 2, ..., M. M is the count of variance differences in observations.For the daily realized volatility of day t, which is between the   and  +1 period, the unconditional variance of it is   2 .  represents the day of the  ℎ structural point.
is used to test whether there exists a structural break, particularly  0 =   = 0. Set the null hypothesis  0 :   2 =  +1 2 and the alternative hypothesis  1 :   2 ≠  +1 2 to test the difference of constant unconditional variances between two independent sample intervals.The CSS is normalized and centered by   .No variance structural break occurs in a time series for a given k when it oscillates about 0, but the variance changes when   deviates from zero.
is defined to find the maximum absolute value of the potential structural break.The  * represents the location in the series of max  |  |.
The null hypothesis claimed may be rejected if   * is greater than the default threshold [8].In this case,  * is set to be the value of k when selecting max  |  |.Then there exists a structural break point near k, if max  |  | is greater than the default threshold.In terms of asymptotic behavior, resembles a Brownian bridge due to its variance homogeneity [10].Based on research by Gong and Lin [11] at the asymptotic 99 ℎ percentile of  = max |, this paper selects 1.628 as the default threshold to estimate.When   is greater than 1.628, it's determined to be a break.The test is held in both the forward and backward order to ensure that the possible turning point is precisely a structural break.1. Dummy variables   (j = 1,2, ..., 17) are set to track how structural breaks affect the data.These 17 dummy variables are set to 0 at the time before the first structural breakdowns, which is chosen as the basis.For   between the  ℎ and the  + 1 ℎ structural breaks,   is set to 1, and others are set to 0.

Day-of-the-week effect and trading volume
S&P 500 trades from Monday to Friday each week.In this article, Monday is used as the foundation, while Tuesday through Friday are represented by the four dummy variables   , where i = 1, 2, 3, and 4. When Tuesday is the trading day,  1 is set to 1 and the other variables are set to zero.The same procedure is followed for Wednesdays, Thursdays, and Fridays.Four of these dummy variables had their values set to zero on Mondays.
Each 5-minute high-frequency trading volume with a unit of 10 million is added together to create the daily trading volume.The daily trading volatility of the  ℎ trading day is   with its formula as followed.
= ∑  ,  =1 (7) where  , is the trading volume in the  ℎ 5-minute trading period of the  ℎ trading day.

HAR-RV models
In this article, the logarithmic HAR-RV model is chosen for prediction.As the influence of dummy variables in modeling is unknown, models of the combination of these variables are formed and the outcomes would be compared, shown as in Table 2. +  ln

Descriptive statistical analysis
The mean of  , ,  , and  , as shown in Table 3. below of the descriptive statistical analysis of describable variables, are on the decline.The maximum value of these three variables exhibits the same tendency, which is the inverse of the trend of the minimum as well as standard deviation.The definition of  , and  , leads to these conclusions of maximum and minimum.The differences between maximum and minimum also follow the decreasing tendency.Though setting the unit of   to be 10 million, both the mean and sd are much greater than the other three variables.Also, the minimum of   is the smallest, while its maximum is the greatest.

In-sample analysis
From table 4., for structural breaks, the significance of   is polarizing. 12 is the most important for the prediction of any period. 7 ,  13 , and  14 are significant for 1-day and 1week prediction. 10 is more significant for 1-month prediction than 1-week and considered to be insignificant for 1-day prediction.The significant distribution of  4 is similar as  10 .In each model relevant to structural breaks, the significance of   various, and for all of them, the number of significant   is less than or equal to half of the total number of   .
For the impact of weekdays,  3 is of importance for mid-term and long-term prediction with a negative coefficient.Also, from the HAR-RV results of mid-and long-term prediction, the coefficients of which are mostly negative, especially for those considered to be significant.The significant   of short-term is  2 and  4 .The coefficients of  2 in 1-day prediction are positive, while those of  4 are always negative.
For all the models shown above and for any period of prediction, trading volume is insignificant.Although the significance and the parameters of the dummy variables have been discussed in the last session, their effects on accuracy are still ambiguous.To determine whether these dummy variables have a beneficial influence on the accuracy of in-sample prediction, adjusted  2 is used to quantify the accuracy of prediction models.From the last line of Table 4, all the models improve the accuracy of 1-day prediction, except for the HAR-RV-Vol model.Considering both structure and the impact of weekdays improve the accuracy of modeling, especially for structural breaks.In mid-term prediction, HAR-RV-SW outperforms the others, and the accuracy of HAR-RV-Vol, which is close to the one of logarithmic HAR-RV, has the lowest accuracy among models with dummy variables.For 1-month prediction, the accuracy of the HAR-RV model is 0.9962, and the models considering structure, weekdays and their combination improve the models' accuracy to 0.9965.

Loss functions
In this session, the loss functions used to assess the prediction models are MAE, MSE, and MAPE.From Table 5., for 1-day prediction, the HAR-RV-Vol model only outperforms the others evaluated using MAE and all the models with dummy variables perform worse than the HAR-RV model measured by MSE.From MAPE, HAR-RV-SB, and HAR-RV-Vol together with their combination models outperform the HAR-RV form.For 1-week prediction, only HAR-RV-WD and HAR-RV-WV outperform measured by MAE and the MSE of HAR-RV is the most well-performed compared with other models when increasing significant digits and none of the models containing dummy variables surpasses HAR-RV model for both mid-and long-term based on MAPE.For 1-month prediction, only the HAR-RV-Vol model was slightly well-performed than the HAR-RV models and none of them perform better on the dimension of MAE.

Diebold Mariano test
DM test [12] is employed to assess the model's correctness in more precision and detail.Its null hypothesis is that the predicting accuracy of the two models is equivalent.In this paper, differences in accuracy are assessed using MAE and MSE.
According to Table 6, the HAR-RV model outperforms the HAR-RV-SB model for any period of prediction, the HAR-RV-SV model exceeds the HAR-RV-SB model and the HAR-RV-SWV model is more accurate than the HAR-RV-SW model.HAR-RV-SB model performs better than HAR-RV-SW for 1 day and week prediction, the HAR-RV-VOL model outperforms HAR-RV-SV, and the HAR-RV-WV model is more accurate than the HAR-RV-SWV model.Using ′ to represent the accuracy of the model.
Three inequations can be formed to show the accurate relationship for 1 day and week:  6., shows no substantial difference between the HAR-RV and HAR-RV-VOL models, which makes it impossible to determine with certainty which model is more accurate.The associations between HAR-RV and HAR-RV-WV models are untested over the short and mid-term, HAR-RV and HAR-RV-SV, and HAR-RV-SWV models are not tested over the long term.
From Table 7. and Table 8., there is no clear relationship between HAR-RV and HAR-RV-WV models for the short-and mid-term and between HAR-RV, HAR-RV-SV, and HAR-RV-SWV models over the long-term.

Conclusion
In this paper, RV forecasting of the S&P 500 by using a dataset with a frequency of five minutes.Employing HAR-RV type modes, the influence of structure, weekdays, and trading volume are evaluated for S&P500.From the definition of the HAR-RV model, three types of RV are calculated.To find the structural breaks, the ICSS algorithm has been used.Dummy variables have been constructed to measure the impacts of three factors.In this article, HAR-RV, HAR-RV-SB, HAR-RV-VW, HAR-RV-Vol, HAR-RV-SW, HAR-RV-WV, HAR-RV-SV, and HAR-RV-SWV models are applied in both in-and out-sample prediction.
Analyzing in-sample prediction, structural breaks and the day-of-the-week impact are crucial to increase the model's accuracy, indicating that significant information to help improve the prediction is contained, while the trading volume with few contributions.However, for out-sample prediction evaluated by MSE, MAE, and MAPE, HAR-RV, HAR-RV-Vol, HAR-RV-SB, and HAR-RV-WD models outperform in some loss functions of different forecasting periods.To further measure the

Fig. 2
Fig. 2 Daily realized volatility and the ±3 s.d.bands around structural breaks

Table 1 .
Structural breaks of the daily RV of S&P 500.

Table 2 .
HAR-RV models with factors and their combination forms

Table 3 .
Descriptive data regarding the variables

Table 4 .
Three Scheme comparing

Table 5 .
Data of loss functions

Table 7 .
Additional short-term and mid-term Diebold Mariano test

Table 8 .
Additional long-term DM test f