Weather Prediction Algorithm vs Stock Market Prediction Algorithm
In the case of weather prediction, physical variables such as temperature, velocity, humidity, and density are available to help predict the next state of the system in some organized scientific way. Although this type of system is nonlinear and chaotic, sophisticated prediction models have been developed—thanks to those measurable data-inputs.
However, in the case of stock trading, the inputs to the stock prices are not directly visible nor measurable. Thus, building stock market prediction algorithm models based on causation is impractical. This is why some entities treat the market as a statistical variant.
Yet we still consider the system to be nonlinear and chaotic—just like the weather. The historical record shows that portfolio returns are still erratic, which means stock market prediction is not yet an established science.
Using Correlations in Stock Market Trading
Trading strategies seem to seek correlations with external events in some way. This choice is tricky, because correlation does not necessarily mean causation. In addition, correlations (when known) are often time-changing (and temporary).
Current thinking seems to favor correlation-based ideas for stock market prediction algorithms. This includes leveraging large databases using artificial intelligence, and machine learning for price prediction. Ostensibly the goal is to improve returns performance. Yet so far, it does not seem to be happening.
For example, AIEQ, the AI powered equity ETF, correlates strongly with the S&P, the market-capitalization-weighted index of the 500 largest U.S. publicly traded companies. AIEQ decisions are 100% machine, while the S&P represents, in some sense, “the broad market.” In our view, such similarity indicates the two have the same systematic flaw in the correlation modeling philosophy.
Millisecond Stock Price Fluctuations Do Not Correlate
Price data can look quite different, depending on the time scales, and the sample rates we choose. For example, we wrote a post about comparing ES and SPY on the intraday time scale:
As the time period goes from long intervals to very short intervals, the correlation between the indexes goes from approximately 1 (strong correlation) to approximately .008 or less (very weak correlation).
For those who want “market neutral,” it is available. For those who want correlation, it is available.
Volatility in stock pricing hides the truth. Moving averages help clean up the noise, but predictive potential requires engineering-strength digital signal processing (DSP) methods.
Example of Digital Signal Processing for Price Prediction
Every equation has a characteristic frequency response. The frequency response informs us in an instant whether the equation will be quantitatively effective for our mission’s goals, or not. In this context, it is not necessary to run Monte Carlo simulations—attempting to model the probability of all different outcomes of a process.
When we say, “characteristic frequency response”, we are literally referring to inherent properties of that equation. This robust de-noising feature is especially useful in messy and noisy systems.
Summing up, whenever a system does not have visible inputs, option B is to look at the data itself for clues. Typically, first you remove the noise in some scientific way. Then, physics principles and DSP can be used to reveal dynamic movements. This is actually a very common practice in aerospace flight test data analysis.
Learn More about Visual Dynamics Examples
On the References tab, we have two links showing “visual” examples of dynamics. Each video shows something about sample rate, or frequency-related dynamics. What we “see” depends on those choices.
Note: total video viewing time is ~15 minutes.