A Stochastic Process Approach in Modelling the Behaviour of HRV as a Biomarker for Different Cognitive States

Jeffery Jonathan Joshua (ישוע) Davis ⁽¹⁾, Florian Schübeler ⁽²⁾

¹ joshua_888@yahoo.com, ² florian@theembassyofpeace.com

The Embassy of Peace, Whitianga, New Zealand

Abstract – This paper is intended to provide a methodology whereby we can compare different signals from Heart Rate Variability (HRV) measurements under different experimental conditions, on six (6) participants, showing different behaviours derived from the different conditions and associated to three (3) conjectured subjective states. We introduce a novel approach based on the Euclidean Distance (ED) index and we test this approach both with simulated and experimental data. The simulated data is generated via Monte Carlo simulation, using the theoretical framework of Markov Chains, while the experimental data is gathered using an emWave device. This methodology, we foresee, will be used in future studies in consciousness and the neurobiology and heart dynamics of spiritual values and their associated psychophysiological states.

Keywords – Heart Rate Variability (HRV), Psychophysiological Coherence, Coherence Scores, Euclidean Distance, Markov Chain Analysis, Inner Peace.

JoMS| 2019 | 1:25 
Reviewed By: Gagandipsingh Khanduja, Electrical Engineering, Marwadi University, Rajkot, India.
Dates: Submission: 20.12.2018 | Acceptance: 15.02.2019 | Publication: 18.02.2019

Download Full Paper

Introduction

The purpose of this paper is to provide a methodology in order to differentiate between three (3) physiological states (coherent, relaxed and stressed) presumably associated to three (3) different subjective states of being (very peaceful, good and survival mode respectively).

The three (3) physiological states are measured using an emWave device and software developed by the HeartMath Institute. This technology monitors heart rate variability (HRV) where the coherent state is displayed in the colour green, the relaxed state in colour blue and the stressed state in colour red.

This study also presents an avenue of describing and modelling the behaviour of HRV in terms of stochastic processes that can be defined as Markov chains in discrete time (Ross, 1985) (Allen, 1978) and simulated using Monte Carlo simulation (Law & Kelton, 1991) in order to model three (3) different HRV modality processes that are represented as three (3) different transition probability matrices as follows:

• A transition probability matrix to represent a dominant coherent process which we call a green transition matrix (G_ij).
• A transition probability matrix to represent a dominant relaxed process which we call a blue transition matrix (B_ij).
• A transition probability matrix to represent a dominant stressed process which we call a red transition matrix (R_ij).

Additionally, we calculated the Euclidean Distance (ED) (sometimes referred to as the Norm₂) as an index used to compare the simulated data vectors for the three (3) different HRV modalities with an ideal vector corresponding to a maximum coherent score. The ED index captures the divergence of each of these three (3) modalities from the ideal and it has been used in different studies of ECoG and EEG brain dynamics as a measurement of distance between signals (Kozma et al., 2012)  (Davis & Kozma, 2012) (Davis & Kozma, 2013).

It is important to note that in real life people display a combination of these three (3) modalities intermittently, according to their awareness and capacity to mediate stressful psychophysiological states, in order to maximize coherent states when the brain processes stimuli coming from the environment to produce meaningful information and knowledge for decision making (Kozma et al., 2012) (Davis & Kozma, 2012) (Davis & Kozma, 2013) (Davis et al., 2015). This suggests that there exists a relationship between HRV and brain dynamics (Thayer et al., 2012), something we intend to address in future studies.

Finally, we apply the ED index to compare experimental data measured with the emWave technology on six (6) participants under two (2) different conditions (baseline and rhythmic breathing).  It is important to note that the effects of rhythmic breathing on HRV for producing coherent states has been well researched and documented (McCraty & Atkinson, 1996) (Leonaite & Vainoras, 2010).

Experimental Methods and Models

We developed a simulation model based on the theoretical framework of Markov chains in discrete time. Then we used stochastic simulation to represent the behaviour of HRV in time, in order to describe three (3) different physiological modalities which we have termed coherent, relaxed and stressed.

A Markov chain in discrete time is a stochastic process which describes the state of a system at certain points in time, where the value of the immediate future state of the system is unaffected by past states and depends only on the present state. The system is usually represented by a probability transition matrix P_ij.

For notational convenience in our case, we will assign the state variable X_t the values i = 0, 1 or 2 where:

• 0 represents a coherent physiological state.
• 1 represents a relaxed physiological state.
• 2 represents a stressed physiological state.

This means that the state of the system at any given time t is X_t and it can take the values X_t = 0, 1 or 2. We can think of the process as making state transitions at times t, where t = 1, 2, 3……N. If X_t = i, the system is said to be in state i at time t. We can suppose that whenever the system is in state i, there is a fixed probability P_ij that it will next be in a state j. P_ij is the one-step transition probability from state i to state j as already mentioned above. The one-step transition probabilities matrix P_ij for this system can be written as:

Where:

• P₀₀ is the probability that given that the system is in a coherent state in the present moment, it will remain in the coherent state in the next immediate moment.
• P₀₁ is the probability that given that the system is in a coherent state in the present moment, it will make a transition to a relaxed state in the next immediate moment.
• P₀₂ is the probability that given that the system is in a coherent state in the present moment, it will make a transition to a stressed state in the next immediate moment.
• P₁₀ is the probability that given that the system is in a relaxed state in the present moment, it will make a transition to a coherent state in the next immediate moment.
• P₁₁ is the probability that given that the system is in a relaxed state in the present moment, it will remain in the relaxed state in the next immediate moment.
• P₁₂ is the probability that given that the system is in a relaxed state in the present moment, it will make a transition to a stressed state in the next immediate moment.
• P₂₀ is the probability that given that the system is in a stressed state in the present moment, it will make a transition to a coherent state in the next immediate moment.
• P₂₁ is the probability that given that the system is in a stressed state in the present moment, it will make a transition to a relaxed state in the next immediate moment.
• P₂₂ is the probability that given that the system is in a stressed state in the present moment, it will remain in the stressed state in the next immediate moment.

The three (3) different conjectured subjective states of being associated with the three (3) different HRV processes are:

• A very peaceful state (predominantly coherent), that we call the coherent modality.
• A good state (predominantly relaxed), that we call the relaxed modality.
• A survival mode state (predominantly stressed), that we call the stressed modality.

These states can be associated to the presence or absence of spiritual values in relationship to cognitive states affecting perception of reality (Childre & McCraty, 2001) (van der Eijk, 2007) (Crivellato & Ribatti, 2007) (Gillett, 2008) (Davis, 2009) (Thayer et al., 2012).

In order to model the state transitions of the system, we need to describe those changes using transition probability matrices (P_ij); one which will generate predominantly coherent states (G_ij), the second one which will generate predominantly relaxed states (B_ij), and the third one which will generate predominantly stressed states (R_ij), for example:

After that, we simulate the system for the three (3) different transition probability matrices as shown in Figure 1 and 2.

Then, we calculate the ED index between each simulation and an ideal vector corresponding to a maximum coherence score which we have termed Ideal Cumulative Score (ICS_t) as shown in Figure 2 and more explicitly in Figure 3.

We expect that the ED index from the ideal score will be different for each transition probability matrix. The variables of the system involved in the comparison of the different modalities are: Ideal Cumulative Score (ICS_t), Euclidean Distance (ED) and Cumulative Score (CS_t). The equations for the different variables of the system are as follows:

where $$ {ICS_t = \sum_{t=1}^{N} X_t~~ Where \thinspace\thinspace X_t = 2 ~\forall ~t} ~~~~ (1) $$

$$ {CS_t = S_0 + \sum_{t=1}^{N} S_t} ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ (2) $$

$$ ED = \sqrt{\sum_{t=1}^{N} (ICS_t - CS_t)^2} ~~~~~~~~~~~~~~~~~ (3) $$

Where:

It is important to note that the system variable S_t stands for the score that is assigned to each physiological state X_t where, for example, if X₃ = 0 then the score S₃ = 2. The reader should observe that coherent states have a greater score than relaxed states and that relaxed states have a greater score than stressed states.

Methodology

In this experiment, a HeartMath emWave device (Quantum Intech, Inc., 2010) was used to monitor six (6) participants, three (3) male and three (3) female adults, under two (2) different experimental conditions in order to obtain HRV data, one at baseline and another whilst performing rhythmic breathing.

The baseline data (Condition 1) was obtained for each participant whilst they sat upright in a chair with both feet flat on the ground, looking out of a window at nature for a period of five (5) minutes. The rhythmic breathing data (Condition 2) was obtained for each participant whilst they lay on their back on a massage table taking regular, deep breaths, with eyes closed for a period of five (5) minutes.

The experiments for each participant were run in succession, baseline first and rhythmic breathing second with a gap of around three (3) minutes. They took place in a lab setting between 5 pm and 9 pm, all in one (1) day, with only natural lighting. Participants were in the room with an experiment supervisor who gave instructions, monitored the session and ran the equipment and software.

Data was collected using the emWave device with a sensor attached to the ear lobe, in order to record the participant’s HRV information on a hard disk. This measure has been classified in three (3) different physiological states which are displayed as: (1) green, a state of high coherence (Coherent), (2) blue, a middle state of coherence (Relaxed) or (3) red, a state of incoherence (Stressed). 

The emWave software automatically calculates and displays a cumulative score which indicates the trajectory of the participant’s HRV in terms of the three (3) different states (coherent, relaxed and stressed). This score is calculated in a similar manner as our CS_t, however, instead of using a Markovian simulation model to generate each state S_t, they are based on a computational algorithm that uses some form of conversion from frequency in Hz to score units for S_t in a fixed window of time of five (5) seconds depending on the type of frequency (VLF, LF, HF) that characterizes the signal in that window. It is important to note that Very Low Frequencies (VLF) are in the interval (0-0.04] Hz, Low Frequencies (LF) are in the interval (0.04-0.15] Hz and High Frequencies (HF) are in the interval (0.15-0.4] Hz and each of them is associated with the different physiological modalities as shown in (McCraty & Atkinson, 1996) (Tarvainen & Niskanen, 2012) (McCraty & Shaffer, 2015).

Analysis of Data

A.   Simulation Results

In Figure 4 we summarize the results obtained in the simulation section where we can appreciate the differences between the values for the ED in each modality (coherent, relaxed and stressed).

As expected, the coherent state shows smaller ED values than the relaxed state which in turn, shows smaller ED values than the stressed state. Now, since the ED index captures the difference between modalities, we are in the position to apply this measure to our experimental data.

B.   Experimental Results

In order to uncover some of the differences between conditions, we show the results for both of them. In Figure 5 we can appreciate that the scores for the different participants in Condition 1 are very diverse. This is expected since every participant will be existing in different baselines at different times, something that requires careful attention in future studies.

For example, when we look at the results  of Participant 1, we observe that he or she is very far from the Ideal Cumulative Score, something that shows that most of the time this participant was in a stressed (red) state. On the other side of the spectrum, the Cumulative Score (CS) of Participant 5 is very close to ICS, which means that apart from the initial period (around 80 seconds) the participant stayed in a coherent (green) state for the rest of the experiment. The rest of the participants, intermittently and at different times, switched from periods of coherence to periods of either relaxed or stressed states and vice versa.

This is a situation that could be studied via simulation, where the main system’s variable (participant’s HRV score) shows a random or cyclical change pattern between the three (3) different transition probability matrices (G_ij, B_ij, R_ij) associated with the three (3) different simulated modalities (coherent, relaxed and stressed respectively). Then we could create a catalogue of different kinds of baseline processes with their corresponding ED, associated in turn, with different cognitive and spiritual subjective states. 

In Figure 6, we can clearly observe a different behaviour compared with Condition 1 for most participants, something we also expected since they were practising rhythmic breathing, a technique that, as mentioned before, has proved to have a positive effect on HRV. In this sense our results confirm this fact. Most participants apart from Participant 4 and 5 display a CS which is associated with coherent (green) states only. Participant 5 was most of the time coherent (green), apart from short periods of time at the beginning and towards the end of the experiment, where he or she showed the stressed (red) state. Participant 4 in Condition 2, on the other hand, showed a similar to baseline (Condition 1) behaviour as Participant 2, 3, 4 and 6 where they alternate between periods of coherent, relaxed and stressed states.

Finally, we present a comparative analysis in Figure 7 where we summarize the results for each participant in each condition. From these graphs it is clear that overall, participants in Condition 2 show a greater tendency towards coherent states than participants in baseline (Condition 1).

It is interesting to observe that Participant 1 and 6 coincide incidentally in showing the same extreme behaviours for both conditions. We observe that in Condition 1 both of them show very large ED values associated with the stressed state while in Condition 2 they both show very small ED values associated with the coherent state.

Conclusions

We have developed a Monte Carlo simulation model that generates different HRV states (coherent, relaxed and stressed) based on the theoretical framework of Markov Chains. The results of the simulations based on these three (3) modalities, have been successfully used to calculate the Euclidean Distance (ED) index for each modality in order to compare them, something that captured the difference between modalities. A more computationally efficient substitute for ED would be the last Cumulative Score (CS_N, where N is the final time t). However, it is important to note that the ED could also be computed based on the state variables (X_t) or their corresponding scores (S_t) in which case it would be an appropriate measure for any stochastic process that has memory. The pros and cons of using this alternative measure of distance (ED) based on X_t or S_t is out of the scope of this paper and is left for future research studies.

We observed as in other studies (Davis et al., 2019) that the practice of rhythmic breathing has a positive effect on HRV, showing increased periods of coherence. We also observed that the baseline state for all participants is very diverse and it more likely depends on the life experience of each participant at different times of the day conditioned by multiple factors. These results though promising should be taken as preliminary since the data sets are very small and this paper is only intended to illustrate methodological tools.

We surely foresee the use of this methodology for further studies in consciousness and the neurobiology of peace and its associated brain and heart dynamics.

Acknowledgements

We would like to acknowledge the team at The Embassy of Peace, in Whitianga, New Zealand for their invaluable participation. Particularly, Maria, Shahar, Matthew, Kali, Colin, Enya, Carey, Sarah, Shiloh, and Keryn.

References

1. Allen, A.O., 1978. Probability, Statistics, and Queueing Theory with Computer Science Applications. 1st ed. Orlando, FL: Academic Press
2. Childre, D. & McCraty, R., 2001. Psychophysiological Correlates of Spiritual Experience. Biofeedback, 29(4), pp.13-17. LINK
3. Crivellato, E. & Ribatti, D., 2007. Soul, mind, brain: Greek Philosophy and the birth of neuroscience. Brain Research Bulletin, 71(4), pp.327-36. LINK
4. Davis, J.J.J., 2009. The Brain of Melchizedek: A Cognitive Neuroscience Approach to Spirituality (Thesis, Master of Science). Dunedin, New Zealand: University of Otago. LINK
5. Davis, J.J.J., Gillett, G. & Kozma, R., 2015. Revisiting Brentano on Consciousness: Striking Correlations with Electrocorticogram Findings about the Action-Perception Cycle and the Emergence of Knowledge and Meaning. Mind and Matter, 13(1), pp.45-69. LINK
6. Davis, J.J. & Kozma, R., 2012. Analysis of phase relationship in ECoG using Hilbert transform and information theoretic measures. In The 2012 International Joint Conference on Neural Networks (IJCNN). Brisbane, IEEE Press. LINK
7. Davis, J.J. & Kozma, R., 2013. Creation of knowledge & meaning manifested via cortical singularities in cognition: Towards a methodology to understand intentionality and critical behavior in neural correlates of awareness. In The 2013 IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain (CCMB). Singapore, IEEE Press. LINK
8. Davis, J.J.J., Schübeler, F. & Kozma, R., 2019. Psychophysiological Coherence in Community Dynamics – A Comparative Analysis between Meditation and Other Activities. OBM Integrative and Complementary Medicine, 4(1), pp. 1-24. LINK
9. Gillett, G.R., 2008. Subjectivity and being somebody: Human identity and Neuroethics. Exeter, UK: Imprint Academic.
10. Kozma, R., Davis, J.J.J. & Freeman, W.J., 2012. Synchronized Minima in ECoG Power at Frequencies Between Beta-Gamma Oscillations Disclose Cortical Singularities in Cognition. Journal of Neuroscience and Neuroengineering, 1(1), pp.13-23. LINK
11. Law, A.M. & Kelton, W.D., 1991. Simulation Modeling and Analysis. 2nd ed. New York: McGraw-Hill
12. Leonaite, A. & Vainoras, A., 2010. Heart Rate Variability during two Relaxation Techniques in Post-MI Men. Electronics and Electrical Engineering, 101(5), pp.107-10. LINK
13. McCraty, R. & Atkinson, M., 1996. Autonomic Assessment Report: A Comprehensive Heart Rate Variability Analysis – Interpretation Guide and Instructions. Boulder Creek, CA: Institute of HeartMath.
14. McCraty, R. & Shaffer, F., 2015. Heart Rate Variability: New Perspectives on Physiological Mechanisms, Assessment of Self-regulatory Capacity, and Health Risk. Global Advances in Health and Medicine, 4(1), pp.46-61. LINK
15. Quantum Intech, Inc., 2010. emWave Desktop Owner's Manual for PC and Mac. [Online] Available at: LINK [Accessed 14 February 2019]
16. Ross, S.M., 1985. Introduction to Probability Models. 3rd ed. Orlando, FL: Academic Press.
17. Tarvainen, M.P. & Niskanen, J.-P., 2012. Kubios HRV version 2.1 User's Guide. Kuopio, Finland: University of Eastern Finland.
18. Thayer, J.F. et al., 2012. A meta-analysis of heart rate variability and neuroimaging studies: Implications for heart rate variability as a marker of stress and health. Neuroscience and Biobehavioral Reviews, 36(2), pp.747–56. LINK
19. van der Eijk, P., 2007. Body and Spirit in Greek medicine and philosophy. [Online] Available at: LINK[Accessed 14 February 2019]