Is it good practice to use Linear Least-Squares with SMA?How to correctly apply a linear trendline...

What is the significance behind "40 days" that often appears in the Bible?

Is "upgrade" the right word to use in this context?

Is there a hypothetical scenario that would make Earth uninhabitable for humans, but not for (the majority of) other animals?

Shortcut for setting origin to vertex

Why does overlay work only on the first tcolorbox?

PTIJ: Who should I vote for? (21st Knesset Edition)

How difficult is it to simply disable/disengage the MCAS on Boeing 737 Max 8 & 9 Aircraft?

What is "focus distance lower/upper" and how is it different from depth of field?

How could an airship be repaired midflight?

What did “the good wine” (τὸν καλὸν οἶνον) mean in John 2:10?

Math equation in non italic font

Is honey really a supersaturated solution? Does heating to un-crystalize redissolve it or melt it?

Recruiter wants very extensive technical details about all of my previous work

This word with a lot of past tenses

As a new Ubuntu desktop 18.04 LTS user, do I need to use ufw for a firewall or is iptables sufficient?

Is it insecure to send a password in a `curl` command?

Are all passive ability checks floors for active ability checks?

What options are left, if Britain cannot decide?

Most cost effective thermostat setting: consistent temperature vs. lowest temperature possible

Why is the President allowed to veto a cancellation of emergency powers?

Why do newer 737s use two different styles of split winglets?

What is a ^ b and (a & b) << 1?

Happy pi day, everyone!

A single argument pattern definition applies to multiple-argument patterns?



Is it good practice to use Linear Least-Squares with SMA?


How to correctly apply a linear trendline equationMeasuring treatment effect on top-ranked subjects selected at point in time from longitudinal dataEnsemble model performs better with worse performing consitutent models?Textbooks on linear regression with least squaresInterpreting regression and $R^2$ with small $n$Solution to force a polynomial curve to end at a specific locationLinear Regression Understanding Least SquaresCan residuals be calculated from N-point moving averages or just the regression line? Also, what is the standard way to determine regression line?Line of best fit does not look like a good fit. Why?Linear least squares algorithms













1












$begingroup$


I have time-series (daily) data and I want to understand the general trend.



My current approach is:




  1. Calculate the 7-day simple moving average.


  2. Add a line of best fit (linear least squares regression).


  3. Plot, then review metrics such as r, r^2, etc.



Question: is it good practice to draw a line-of-best fit on a moving average? I'm not very experienced but my understanding is MA and linear trend lines are both trend lines, so I'm not sure if it's OK to combine them in this way.



Raw data looks like this:



day             + count

2015-01-01 | 123

2015-01-02 | 290

2015-01-03 | 329

2015-01-04 | 276



Let me know if more detail would help- any direction on this is much appreciated.






regression time-series correlation trend moving-average







share|cite|improve this question







New contributor




Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.







$endgroup$








  • 1




    $begingroup$
    It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
    $endgroup$
    – whuber♦
    2 hours ago


















1












$begingroup$


I have time-series (daily) data and I want to understand the general trend.



My current approach is:




  1. Calculate the 7-day simple moving average.


  2. Add a line of best fit (linear least squares regression).


  3. Plot, then review metrics such as r, r^2, etc.



Question: is it good practice to draw a line-of-best fit on a moving average? I'm not very experienced but my understanding is MA and linear trend lines are both trend lines, so I'm not sure if it's OK to combine them in this way.



Raw data looks like this:



day             + count

2015-01-01 | 123

2015-01-02 | 290

2015-01-03 | 329

2015-01-04 | 276



Let me know if more detail would help- any direction on this is much appreciated.






regression time-series correlation trend moving-average







share|cite|improve this question







New contributor




Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.







$endgroup$








  • 1




    $begingroup$
    It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
    $endgroup$
    – whuber♦
    2 hours ago
















1












1








1





$begingroup$


I have time-series (daily) data and I want to understand the general trend.



My current approach is:




  1. Calculate the 7-day simple moving average.


  2. Add a line of best fit (linear least squares regression).


  3. Plot, then review metrics such as r, r^2, etc.



Question: is it good practice to draw a line-of-best fit on a moving average? I'm not very experienced but my understanding is MA and linear trend lines are both trend lines, so I'm not sure if it's OK to combine them in this way.



Raw data looks like this:



day             + count

2015-01-01 | 123

2015-01-02 | 290

2015-01-03 | 329

2015-01-04 | 276



Let me know if more detail would help- any direction on this is much appreciated.






regression time-series correlation trend moving-average







share|cite|improve this question







New contributor




Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.







$endgroup$




I have time-series (daily) data and I want to understand the general trend.



My current approach is:




  1. Calculate the 7-day simple moving average.


  2. Add a line of best fit (linear least squares regression).


  3. Plot, then review metrics such as r, r^2, etc.



Question: is it good practice to draw a line-of-best fit on a moving average? I'm not very experienced but my understanding is MA and linear trend lines are both trend lines, so I'm not sure if it's OK to combine them in this way.



Raw data looks like this:



day             + count

2015-01-01 | 123

2015-01-02 | 290

2015-01-03 | 329

2015-01-04 | 276



Let me know if more detail would help- any direction on this is much appreciated.






regression time-series correlation trend moving-average




regression time-series correlation trend moving-average






share|cite|improve this question







New contributor




Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











share|cite|improve this question







New contributor




Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









share|cite|improve this question




share|cite|improve this question






New contributor




Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









asked 2 hours ago









Chef36Chef36

61




61




New contributor




Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.





New contributor





Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.






Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.








  • 1




    $begingroup$
    It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
    $endgroup$
    – whuber♦
    2 hours ago
















  • 1




    $begingroup$
    It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
    $endgroup$
    – whuber♦
    2 hours ago










1




1




$begingroup$
It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
$endgroup$
– whuber♦
2 hours ago






$begingroup$
It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
$endgroup$
– whuber♦
2 hours ago












1 Answer
1






active

oldest

votes


















2












$begingroup$

Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.



Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.



These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.






share|cite|improve this answer









$endgroup$













    Your Answer





    StackExchange.ifUsing("editor", function () {
    return StackExchange.using("mathjaxEditing", function () {
    StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
    StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
    });
    });
    }, "mathjax-editing");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "65"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: false,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });






    Chef36 is a new contributor. Be nice, and check out our Code of Conduct.










    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f397917%2fis-it-good-practice-to-use-linear-least-squares-with-sma%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    2












    $begingroup$

    Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.



    Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.



    These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.






    share|cite|improve this answer









    $endgroup$


















      2












      $begingroup$

      Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.



      Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.



      These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.






      share|cite|improve this answer









      $endgroup$
















        2












        2








        2





        $begingroup$

        Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.



        Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.



        These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.






        share|cite|improve this answer









        $endgroup$



        Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.



        Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.



        These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.







        share|cite|improve this answer












        share|cite|improve this answer



        share|cite|improve this answer










        answered 2 hours ago









        Gkhan CebsGkhan Cebs

        1443




        1443






















            Chef36 is a new contributor. Be nice, and check out our Code of Conduct.










            draft saved

            draft discarded


















            Chef36 is a new contributor. Be nice, and check out our Code of Conduct.













            Chef36 is a new contributor. Be nice, and check out our Code of Conduct.












            Chef36 is a new contributor. Be nice, and check out our Code of Conduct.
















            Thanks for contributing an answer to Cross Validated!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid …



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            Use MathJax to format equations. MathJax reference.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f397917%2fis-it-good-practice-to-use-linear-least-squares-with-sma%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            -

            Popular posts from this blog

            Why do type traits not work with types in namespace scope?What are POD types in C++?Why can templates only be...

            Image
            What does it mean to express a gate in Dirac notation? A Strange Latex Symbol How can Republicans who favour free markets, consistently express anger when they don't like the outcome of that choice? What happened to Captain America in Endgame? How to solve constants out of the internal energy equation? How can I place the product on a social media post better? Will tsunami waves travel forever if there was no land? Unexpected email from Yorkshire Bank What is the relationship between spectral sequences and obstruction theory? How can I practically buy stocks? Will a top journal at least re...
            Read more

            Will tsunami waves travel forever if there was no land?Why do tsunami waves begin with the water flowing away...

            Image
            How much cash can I safely carry into the USA and avoid civil forfeiture? Is there really no use for MD5 anymore? Is contacting this expert in the field something acceptable or would it be discourteous? How to pronounce 'C++' in Spanish How to creep the reader out with what seems like a normal person? Critique of timeline aesthetic Sci fi novel series with instant travel between planets through gates. A river runs through the gates How did Captain America manage to do this? Are Boeing 737-800’s grounded? Error message with tabularx How to make a pipeline wait for end-of-file or stop af...
            Read more

            Simple Scan not detecting my scanner (Brother DCP-7055W)Brother MFC-L2700DW printer can print, can't...

            Image
            Can someone publish a story that happened to you? Will tsunami waves travel forever if there was no land? What is the strongest case that can be made in favour of the UK regaining some control over fishing policy after Brexit? Controversial area of mathematics Packing rectangles: Does rotation ever help? Is there a way to get a compiler for the original B programming language? Using a Lyapunov function to classify stability and sketching a phase portrait Do I have an "anti-research" personality? Reducing vertical space in stackrel What does KSP mean? Pulling the rope with one hand is...
            Read more