Numerical (Second) Derivative of Time Series Data
Clash Royale CLAN TAG#URR8PPP
up vote
1
down vote
favorite
First and second order derivatives are often used in chromatography to detect hidden peaks. The time series data consists of Instrumental Response vs. Time at very short time intervals (250 Hz). I wanted to calculate the second derivative of the data numerically in Excel. The simple option is that we calculate the first derivative and then calculate the first derivative of the first derivative to get the second derivative.
The other option is to use the direct approach using central difference formula for the second derivative. The question is about the denominator of the second derivative from the central difference formula. It should the square of the time interval. This is my understanding and it is consistent dimensionally for example distance x (m) becomes acceleration (m/s2) as the second derivative of x.
A reviewer wrote a rather denigrating comment saying that there is a lack of understanding of the second derivative "definition" where the authors assert that the definition of a second derivative requires division by the square of the time interval. This reference to the square of a time interval suggests a worrying lack of understanding of the nature of the derivative ðÂ’…ðŸÂ/ðÂ’…ðÂ’•ðŸ as an operator and not as an algebraic variable.
Do mathematicians agree with the above comment? Can we interpret d^2/dt^2 as if it is repeating the d operator twice divided by time interval squared? Thanks.
calculus discrete-mathematics derivatives operator-theory
asked Aug 7 at 17:14
M. Farooq
83
 |Â
show 3 more comments
up vote
1
down vote
favorite
First and second order derivatives are often used in chromatography to detect hidden peaks. The time series data consists of Instrumental Response vs. Time at very short time intervals (250 Hz). I wanted to calculate the second derivative of the data numerically in Excel. The simple option is that we calculate the first derivative and then calculate the first derivative of the first derivative to get the second derivative.
The other option is to use the direct approach using central difference formula for the second derivative. The question is about the denominator of the second derivative from the central difference formula. It should the square of the time interval. This is my understanding and it is consistent dimensionally for example distance x (m) becomes acceleration (m/s2) as the second derivative of x.
A reviewer wrote a rather denigrating comment saying that there is a lack of understanding of the second derivative "definition" where the authors assert that the definition of a second derivative requires division by the square of the time interval. This reference to the square of a time interval suggests a worrying lack of understanding of the nature of the derivative ðÂ’…ðŸÂ/ðÂ’…ðÂ’•ðŸ as an operator and not as an algebraic variable.
Do mathematicians agree with the above comment? Can we interpret d^2/dt^2 as if it is repeating the d operator twice divided by time interval squared? Thanks.
calculus discrete-mathematics derivatives operator-theory
asked Aug 7 at 17:14
M. Farooq
83
Just point the reviewer to en.wikipedia.org/wiki/…
– Andrei
Aug 7 at 17:26
If the second derivative of $f$ exists, then $f''(x)=lim_hto 0 fracDelta_h^2 f(x)h^2$, where $Delta_h f(x) = f(x+h)-f(x)$, so we can approximate the second derivative this way
– Rumpelstiltskin
Aug 7 at 17:27
Similarly, if the $n$-th derivative of $f$ exists, then $f^(n)(x) = lim_hto 0 fracDelta^n_hf(x)h^n$. Note that, even if the limit exists, this doesn't guarantee existence of the $n$-th derivative. For this result, you can look in the book from "Grigorii Mikhailovich Fichtenholz", which, according to wikipedia, unfortunately isn't available in English.
– Rumpelstiltskin
Aug 7 at 17:37
@Rumpelstiltskin, I'm afraid your limit will converge to $f'^2$ instead that $f"$.
– N74
Aug 7 at 18:45
@N74 No, the formula is good. Example, $f(x)=x^3$. $lim_hto 0 frac(x+2h)^3-2(x+h)^3+x^3h^2 = lim_hto 0 6h+6x = 6x$. As expected, $f''(x) = 6x$
– Rumpelstiltskin
Aug 7 at 19:18
 |Â
show 3 more comments
up vote
1
down vote
favorite
up vote
1
down vote
favorite
First and second order derivatives are often used in chromatography to detect hidden peaks. The time series data consists of Instrumental Response vs. Time at very short time intervals (250 Hz). I wanted to calculate the second derivative of the data numerically in Excel. The simple option is that we calculate the first derivative and then calculate the first derivative of the first derivative to get the second derivative.
The other option is to use the direct approach using central difference formula for the second derivative. The question is about the denominator of the second derivative from the central difference formula. It should the square of the time interval. This is my understanding and it is consistent dimensionally for example distance x (m) becomes acceleration (m/s2) as the second derivative of x.
A reviewer wrote a rather denigrating comment saying that there is a lack of understanding of the second derivative "definition" where the authors assert that the definition of a second derivative requires division by the square of the time interval. This reference to the square of a time interval suggests a worrying lack of understanding of the nature of the derivative ðÂ’…ðŸÂ/ðÂ’…ðÂ’•ðŸ as an operator and not as an algebraic variable.
Do mathematicians agree with the above comment? Can we interpret d^2/dt^2 as if it is repeating the d operator twice divided by time interval squared? Thanks.
calculus discrete-mathematics derivatives operator-theory
asked Aug 7 at 17:14
M. Farooq
83
First and second order derivatives are often used in chromatography to detect hidden peaks. The time series data consists of Instrumental Response vs. Time at very short time intervals (250 Hz). I wanted to calculate the second derivative of the data numerically in Excel. The simple option is that we calculate the first derivative and then calculate the first derivative of the first derivative to get the second derivative.
The other option is to use the direct approach using central difference formula for the second derivative. The question is about the denominator of the second derivative from the central difference formula. It should the square of the time interval. This is my understanding and it is consistent dimensionally for example distance x (m) becomes acceleration (m/s2) as the second derivative of x.
A reviewer wrote a rather denigrating comment saying that there is a lack of understanding of the second derivative "definition" where the authors assert that the definition of a second derivative requires division by the square of the time interval. This reference to the square of a time interval suggests a worrying lack of understanding of the nature of the derivative ðÂ’…ðŸÂ/ðÂ’…ðÂ’•ðŸ as an operator and not as an algebraic variable.
Do mathematicians agree with the above comment? Can we interpret d^2/dt^2 as if it is repeating the d operator twice divided by time interval squared? Thanks.
calculus discrete-mathematics derivatives operator-theory
asked Aug 7 at 17:14
M. Farooq
83
asked Aug 7 at 17:14
M. Farooq
83
asked Aug 7 at 17:14
M. Farooq
83
asked Aug 7 at 17:14
M. Farooq
83
83
Just point the reviewer to en.wikipedia.org/wiki/…
– Andrei
Aug 7 at 17:26
If the second derivative of $f$ exists, then $f''(x)=lim_hto 0 fracDelta_h^2 f(x)h^2$, where $Delta_h f(x) = f(x+h)-f(x)$, so we can approximate the second derivative this way
– Rumpelstiltskin
Aug 7 at 17:27
Similarly, if the $n$-th derivative of $f$ exists, then $f^(n)(x) = lim_hto 0 fracDelta^n_hf(x)h^n$. Note that, even if the limit exists, this doesn't guarantee existence of the $n$-th derivative. For this result, you can look in the book from "Grigorii Mikhailovich Fichtenholz", which, according to wikipedia, unfortunately isn't available in English.
– Rumpelstiltskin
Aug 7 at 17:37
@Rumpelstiltskin, I'm afraid your limit will converge to $f'^2$ instead that $f"$.
– N74
Aug 7 at 18:45
@N74 No, the formula is good. Example, $f(x)=x^3$. $lim_hto 0 frac(x+2h)^3-2(x+h)^3+x^3h^2 = lim_hto 0 6h+6x = 6x$. As expected, $f''(x) = 6x$
– Rumpelstiltskin
Aug 7 at 19:18
 |Â
show 3 more comments
Just point the reviewer to en.wikipedia.org/wiki/…
– Andrei
Aug 7 at 17:26
If the second derivative of $f$ exists, then $f''(x)=lim_hto 0 fracDelta_h^2 f(x)h^2$, where $Delta_h f(x) = f(x+h)-f(x)$, so we can approximate the second derivative this way
– Rumpelstiltskin
Aug 7 at 17:27
Similarly, if the $n$-th derivative of $f$ exists, then $f^(n)(x) = lim_hto 0 fracDelta^n_hf(x)h^n$. Note that, even if the limit exists, this doesn't guarantee existence of the $n$-th derivative. For this result, you can look in the book from "Grigorii Mikhailovich Fichtenholz", which, according to wikipedia, unfortunately isn't available in English.
– Rumpelstiltskin
Aug 7 at 17:37
@Rumpelstiltskin, I'm afraid your limit will converge to $f'^2$ instead that $f"$.
– N74
Aug 7 at 18:45
@N74 No, the formula is good. Example, $f(x)=x^3$. $lim_hto 0 frac(x+2h)^3-2(x+h)^3+x^3h^2 = lim_hto 0 6h+6x = 6x$. As expected, $f''(x) = 6x$
– Rumpelstiltskin
Aug 7 at 19:18
Just point the reviewer to en.wikipedia.org/wiki/…
– Andrei
Aug 7 at 17:26
Just point the reviewer to en.wikipedia.org/wiki/…
– Andrei
Aug 7 at 17:26
If the second derivative of $f$ exists, then $f''(x)=lim_hto 0 fracDelta_h^2 f(x)h^2$, where $Delta_h f(x) = f(x+h)-f(x)$, so we can approximate the second derivative this way
– Rumpelstiltskin
Aug 7 at 17:27
If the second derivative of $f$ exists, then $f''(x)=lim_hto 0 fracDelta_h^2 f(x)h^2$, where $Delta_h f(x) = f(x+h)-f(x)$, so we can approximate the second derivative this way
– Rumpelstiltskin
Aug 7 at 17:27
Similarly, if the $n$-th derivative of $f$ exists, then $f^(n)(x) = lim_hto 0 fracDelta^n_hf(x)h^n$. Note that, even if the limit exists, this doesn't guarantee existence of the $n$-th derivative. For this result, you can look in the book from "Grigorii Mikhailovich Fichtenholz", which, according to wikipedia, unfortunately isn't available in English.
– Rumpelstiltskin
Aug 7 at 17:37
Similarly, if the $n$-th derivative of $f$ exists, then $f^(n)(x) = lim_hto 0 fracDelta^n_hf(x)h^n$. Note that, even if the limit exists, this doesn't guarantee existence of the $n$-th derivative. For this result, you can look in the book from "Grigorii Mikhailovich Fichtenholz", which, according to wikipedia, unfortunately isn't available in English.
– Rumpelstiltskin
Aug 7 at 17:37
@Rumpelstiltskin, I'm afraid your limit will converge to $f'^2$ instead that $f"$.
– N74
Aug 7 at 18:45
@Rumpelstiltskin, I'm afraid your limit will converge to $f'^2$ instead that $f"$.
– N74
Aug 7 at 18:45
@N74 No, the formula is good. Example, $f(x)=x^3$. $lim_hto 0 frac(x+2h)^3-2(x+h)^3+x^3h^2 = lim_hto 0 6h+6x = 6x$. As expected, $f''(x) = 6x$
– Rumpelstiltskin
Aug 7 at 19:18
@N74 No, the formula is good. Example, $f(x)=x^3$. $lim_hto 0 frac(x+2h)^3-2(x+h)^3+x^3h^2 = lim_hto 0 6h+6x = 6x$. As expected, $f''(x) = 6x$
– Rumpelstiltskin
Aug 7 at 19:18
 |Â
show 3 more comments
1 Answer
1
active
oldest
votes
up vote
0
down vote
For a series of discrete values $x_1$, $x_2$, .. $x_n$ at regular intervals of time $h$, the central derivate at $x_i$ is
$$ dotx_i = frac x_i+1 - x_i-1 2 h $$
This is preferable because $fracx_i+1-x_ih$ is the average slope of the following interval and not exactly at the point $i$.
To get the second derivative apply the above to the first derivative
$$ beginaligned
ddotx_i & = frac dotx_i+1 - dotx_i-1 2 h \
& = fracfrac x_i+2 - x_i 2 h - frac x_i - x_i-2 2 h 2 h \ & = fracx_i+2+x_i-2-2x_i4 ,h^2
endaligned$$
The above definition is related to the Verlet Integration, also known as the leap-frog method since the derivative at $i$ does not contain the value $x_i$.
answered Aug 7 at 18:30
ja72
7,17711641
Thanks, you mentioned, "...the leap-frog method since the derivative at i does not contain the value xi." In the expression for the second derivative, there is a 2xi term but not in the first derivative.
– M. Farooq
Aug 7 at 18:42
Yes, the second derivative describes curvature and thus the central value is needed. But for the first derivative, due to the fundamental theorem of calculus, the slope depends on the endpoints only.
– ja72
Aug 8 at 12:12
add a comment |Â
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
For a series of discrete values $x_1$, $x_2$, .. $x_n$ at regular intervals of time $h$, the central derivate at $x_i$ is
$$ dotx_i = frac x_i+1 - x_i-1 2 h $$
This is preferable because $fracx_i+1-x_ih$ is the average slope of the following interval and not exactly at the point $i$.
To get the second derivative apply the above to the first derivative
$$ beginaligned
ddotx_i & = frac dotx_i+1 - dotx_i-1 2 h \
& = fracfrac x_i+2 - x_i 2 h - frac x_i - x_i-2 2 h 2 h \ & = fracx_i+2+x_i-2-2x_i4 ,h^2
endaligned$$
The above definition is related to the Verlet Integration, also known as the leap-frog method since the derivative at $i$ does not contain the value $x_i$.
answered Aug 7 at 18:30
ja72
7,17711641
Thanks, you mentioned, "...the leap-frog method since the derivative at i does not contain the value xi." In the expression for the second derivative, there is a 2xi term but not in the first derivative.
– M. Farooq
Aug 7 at 18:42
Yes, the second derivative describes curvature and thus the central value is needed. But for the first derivative, due to the fundamental theorem of calculus, the slope depends on the endpoints only.
– ja72
Aug 8 at 12:12
add a comment |Â
up vote
0
down vote
For a series of discrete values $x_1$, $x_2$, .. $x_n$ at regular intervals of time $h$, the central derivate at $x_i$ is
$$ dotx_i = frac x_i+1 - x_i-1 2 h $$
This is preferable because $fracx_i+1-x_ih$ is the average slope of the following interval and not exactly at the point $i$.
To get the second derivative apply the above to the first derivative
$$ beginaligned
ddotx_i & = frac dotx_i+1 - dotx_i-1 2 h \
& = fracfrac x_i+2 - x_i 2 h - frac x_i - x_i-2 2 h 2 h \ & = fracx_i+2+x_i-2-2x_i4 ,h^2
endaligned$$
The above definition is related to the Verlet Integration, also known as the leap-frog method since the derivative at $i$ does not contain the value $x_i$.
answered Aug 7 at 18:30
ja72
7,17711641
Thanks, you mentioned, "...the leap-frog method since the derivative at i does not contain the value xi." In the expression for the second derivative, there is a 2xi term but not in the first derivative.
– M. Farooq
Aug 7 at 18:42
Yes, the second derivative describes curvature and thus the central value is needed. But for the first derivative, due to the fundamental theorem of calculus, the slope depends on the endpoints only.
– ja72
Aug 8 at 12:12
add a comment |Â
up vote
0
down vote
up vote
0
down vote
For a series of discrete values $x_1$, $x_2$, .. $x_n$ at regular intervals of time $h$, the central derivate at $x_i$ is
$$ dotx_i = frac x_i+1 - x_i-1 2 h $$
This is preferable because $fracx_i+1-x_ih$ is the average slope of the following interval and not exactly at the point $i$.
To get the second derivative apply the above to the first derivative
$$ beginaligned
ddotx_i & = frac dotx_i+1 - dotx_i-1 2 h \
& = fracfrac x_i+2 - x_i 2 h - frac x_i - x_i-2 2 h 2 h \ & = fracx_i+2+x_i-2-2x_i4 ,h^2
endaligned$$
The above definition is related to the Verlet Integration, also known as the leap-frog method since the derivative at $i$ does not contain the value $x_i$.
answered Aug 7 at 18:30
ja72
7,17711641
For a series of discrete values $x_1$, $x_2$, .. $x_n$ at regular intervals of time $h$, the central derivate at $x_i$ is
$$ dotx_i = frac x_i+1 - x_i-1 2 h $$
This is preferable because $fracx_i+1-x_ih$ is the average slope of the following interval and not exactly at the point $i$.
To get the second derivative apply the above to the first derivative
$$ beginaligned
ddotx_i & = frac dotx_i+1 - dotx_i-1 2 h \
& = fracfrac x_i+2 - x_i 2 h - frac x_i - x_i-2 2 h 2 h \ & = fracx_i+2+x_i-2-2x_i4 ,h^2
endaligned$$
The above definition is related to the Verlet Integration, also known as the leap-frog method since the derivative at $i$ does not contain the value $x_i$.
answered Aug 7 at 18:30
ja72
7,17711641
answered Aug 7 at 18:30
ja72
7,17711641
answered Aug 7 at 18:30
ja72
7,17711641
answered Aug 7 at 18:30
ja72
7,17711641
7,17711641
Thanks, you mentioned, "...the leap-frog method since the derivative at i does not contain the value xi." In the expression for the second derivative, there is a 2xi term but not in the first derivative.
– M. Farooq
Aug 7 at 18:42
Yes, the second derivative describes curvature and thus the central value is needed. But for the first derivative, due to the fundamental theorem of calculus, the slope depends on the endpoints only.
– ja72
Aug 8 at 12:12
add a comment |Â
Thanks, you mentioned, "...the leap-frog method since the derivative at i does not contain the value xi." In the expression for the second derivative, there is a 2xi term but not in the first derivative.
– M. Farooq
Aug 7 at 18:42
Yes, the second derivative describes curvature and thus the central value is needed. But for the first derivative, due to the fundamental theorem of calculus, the slope depends on the endpoints only.
– ja72
Aug 8 at 12:12
Thanks, you mentioned, "...the leap-frog method since the derivative at i does not contain the value xi." In the expression for the second derivative, there is a 2xi term but not in the first derivative.
– M. Farooq
Aug 7 at 18:42
Thanks, you mentioned, "...the leap-frog method since the derivative at i does not contain the value xi." In the expression for the second derivative, there is a 2xi term but not in the first derivative.
– M. Farooq
Aug 7 at 18:42
Yes, the second derivative describes curvature and thus the central value is needed. But for the first derivative, due to the fundamental theorem of calculus, the slope depends on the endpoints only.
– ja72
Aug 8 at 12:12
Yes, the second derivative describes curvature and thus the central value is needed. But for the first derivative, due to the fundamental theorem of calculus, the slope depends on the endpoints only.
– ja72
Aug 8 at 12:12
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2875173%2fnumerical-second-derivative-of-time-series-data%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Just point the reviewer to en.wikipedia.org/wiki/…
– Andrei
Aug 7 at 17:26
If the second derivative of $f$ exists, then $f''(x)=lim_hto 0 fracDelta_h^2 f(x)h^2$, where $Delta_h f(x) = f(x+h)-f(x)$, so we can approximate the second derivative this way
– Rumpelstiltskin
Aug 7 at 17:27
Similarly, if the $n$-th derivative of $f$ exists, then $f^(n)(x) = lim_hto 0 fracDelta^n_hf(x)h^n$. Note that, even if the limit exists, this doesn't guarantee existence of the $n$-th derivative. For this result, you can look in the book from "Grigorii Mikhailovich Fichtenholz", which, according to wikipedia, unfortunately isn't available in English.
– Rumpelstiltskin
Aug 7 at 17:37
@Rumpelstiltskin, I'm afraid your limit will converge to $f'^2$ instead that $f"$.
– N74
Aug 7 at 18:45
@N74 No, the formula is good. Example, $f(x)=x^3$. $lim_hto 0 frac(x+2h)^3-2(x+h)^3+x^3h^2 = lim_hto 0 6h+6x = 6x$. As expected, $f''(x) = 6x$
– Rumpelstiltskin
Aug 7 at 19:18