Using wget as a cron job to fetch contents of a PDF daily Unicorn Meta Zoo #1: Why another...
Can you stand up from being prone using Skirmisher outside of your turn?
Co-worker works way more than he should
Trumpet valves, lengths, and pitch
Putting Ant-Man on house arrest
My bank got bought out, am I now going to have to start filing tax returns in a different state?
Married in secret, can marital status in passport be changed at a later date?
Will I lose my paid in full property
A Paper Record is What I Hamper
What do you call the part of a novel that is not dialog?
Rolling Stones Sway guitar solo chord function
How do I check if a string is entirely made of the same substring?
Would reducing the reference voltage of an ADC have any effect on accuracy?
Contradiction proof for inequality of P and NP?
Arriving in Atlanta after US Preclearance in Dublin. Will I go through TSA security in Atlanta to transfer to a connecting flight?
How to translate "red flag" into Spanish?
Does Feeblemind produce an ongoing magical effect that can be dispelled?
Could Neutrino technically as side-effect, incentivize centralization of the bitcoin network?
Israeli soda type drink
Is it acceptable to use working hours to read general interest books?
Suing a Police Officer Instead of the Police Department
A Dictionary or Encyclopedia of Fantasy or Fairy Tales from the 1960s
"Rubric" as meaning "signature" or "personal mark" -- is this accepted usage?
How to open locks without disable device?
What to do with someone that cheated their way through university and a PhD program?
Using wget as a cron job to fetch contents of a PDF daily
Unicorn Meta Zoo #1: Why another podcast?
Announcing the arrival of Valued Associate #679: Cesar ManaraDaily cron job not runningBackup regimen - is this plan any good?Cron job daily not runningDaily Cron job not executingOne single cron job not runningDaily Cron Job killing Java ApplicationDisk Space Shortage since cron job setupUbuntu 16.04LTS Create Cron Job in cron.daily that will display the contents of auth.log to folder in every users home directory?Cron job to move directory with yesterday's date to another directoryCron job daily folder report
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}
I am wanting to fetch the contents of a PDF which is updated daily. I would like to set this as a cron job and have it fetch AND date the PDF file name (daily) so I always have a copy of each days PDF VERSION-COPY.
So:
- Cron runs each day
- Downloads PDF from server
- Copies PDF to disk at "/user/site-xxx"
- Prints PDF file name OR FOLDER NAME according to system date.
This should allow me to retain a copy of each days PDF contents, dated by folder/file name for indexing and searching.
Is this even possible with wget? Feedback is appreciated!
cron pdf wget
edited 6 hours ago
Philippe Delteil
8161722
asked 13 hours ago
jau2019jau2019
11
New contributor
jau2019 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
I am wanting to fetch the contents of a PDF which is updated daily. I would like to set this as a cron job and have it fetch AND date the PDF file name (daily) so I always have a copy of each days PDF VERSION-COPY.
So:
- Cron runs each day
- Downloads PDF from server
- Copies PDF to disk at "/user/site-xxx"
- Prints PDF file name OR FOLDER NAME according to system date.
This should allow me to retain a copy of each days PDF contents, dated by folder/file name for indexing and searching.
Is this even possible with wget? Feedback is appreciated!
cron pdf wget
edited 6 hours ago
Philippe Delteil
8161722
asked 13 hours ago
jau2019jau2019
11
New contributor
jau2019 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Yea, I don't see this as being too hard with a little scripting. As to your question; I have a folder named DAILY where I would like to run this script from, using cron, to automatically: 1. Run Script. 2. Name downloaded file according to the days Date, NEVER overwriting the previous days VERSION. I want the pdfs to be searchable for research purposes. Naming the PDF file names is great, OR have wget/curl instead place each days download into a FOLDER according to the days date, would be fine.
– jau2019
13 hours ago
/user/site-xxx/DAILY
– jau2019
13 hours ago
add a comment |
I am wanting to fetch the contents of a PDF which is updated daily. I would like to set this as a cron job and have it fetch AND date the PDF file name (daily) so I always have a copy of each days PDF VERSION-COPY.
So:
- Cron runs each day
- Downloads PDF from server
- Copies PDF to disk at "/user/site-xxx"
- Prints PDF file name OR FOLDER NAME according to system date.
This should allow me to retain a copy of each days PDF contents, dated by folder/file name for indexing and searching.
Is this even possible with wget? Feedback is appreciated!
cron pdf wget
edited 6 hours ago
Philippe Delteil
8161722
asked 13 hours ago
jau2019jau2019
11
New contributor
jau2019 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
I am wanting to fetch the contents of a PDF which is updated daily. I would like to set this as a cron job and have it fetch AND date the PDF file name (daily) so I always have a copy of each days PDF VERSION-COPY.
So:
- Cron runs each day
- Downloads PDF from server
- Copies PDF to disk at "/user/site-xxx"
- Prints PDF file name OR FOLDER NAME according to system date.
This should allow me to retain a copy of each days PDF contents, dated by folder/file name for indexing and searching.
Is this even possible with wget? Feedback is appreciated!
cron pdf wget
cron pdf wget
edited 6 hours ago
Philippe Delteil
8161722
asked 13 hours ago
jau2019jau2019
11
New contributor
jau2019 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited 6 hours ago
Philippe Delteil
8161722
asked 13 hours ago
jau2019jau2019
11
New contributor
jau2019 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited 6 hours ago
Philippe Delteil
8161722
edited 6 hours ago
Philippe Delteil
8161722
edited 6 hours ago
Philippe Delteil
8161722
8161722
asked 13 hours ago
jau2019jau2019
11
New contributor
jau2019 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 13 hours ago
jau2019jau2019
11
asked 13 hours ago
jau2019jau2019
11
11
New contributor
jau2019 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
jau2019 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
jau2019 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Yea, I don't see this as being too hard with a little scripting. As to your question; I have a folder named DAILY where I would like to run this script from, using cron, to automatically: 1. Run Script. 2. Name downloaded file according to the days Date, NEVER overwriting the previous days VERSION. I want the pdfs to be searchable for research purposes. Naming the PDF file names is great, OR have wget/curl instead place each days download into a FOLDER according to the days date, would be fine.
– jau2019
13 hours ago
/user/site-xxx/DAILY
– jau2019
13 hours ago
add a comment |
Yea, I don't see this as being too hard with a little scripting. As to your question; I have a folder named DAILY where I would like to run this script from, using cron, to automatically: 1. Run Script. 2. Name downloaded file according to the days Date, NEVER overwriting the previous days VERSION. I want the pdfs to be searchable for research purposes. Naming the PDF file names is great, OR have wget/curl instead place each days download into a FOLDER according to the days date, would be fine.
– jau2019
13 hours ago
/user/site-xxx/DAILY
– jau2019
13 hours ago
Yea, I don't see this as being too hard with a little scripting. As to your question; I have a folder named DAILY where I would like to run this script from, using cron, to automatically: 1. Run Script. 2. Name downloaded file according to the days Date, NEVER overwriting the previous days VERSION. I want the pdfs to be searchable for research purposes. Naming the PDF file names is great, OR have wget/curl instead place each days download into a FOLDER according to the days date, would be fine.
– jau2019
13 hours ago
Yea, I don't see this as being too hard with a little scripting. As to your question; I have a folder named DAILY where I would like to run this script from, using cron, to automatically: 1. Run Script. 2. Name downloaded file according to the days Date, NEVER overwriting the previous days VERSION. I want the pdfs to be searchable for research purposes. Naming the PDF file names is great, OR have wget/curl instead place each days download into a FOLDER according to the days date, would be fine.
– jau2019
13 hours ago
/user/site-xxx/DAILY
– jau2019
13 hours ago
/user/site-xxx/DAILY
– jau2019
13 hours ago
add a comment |
1 Answer
1
active
oldest
votes
Edit cron jobs with crontab -e. Add this command:
0 3 * * * /usr/bin/wget --output-file="/pathtofolder/wget.log" --output-document="/pathtofolder/Document$(date +%H-%M-%Y-%m-%d).pdf" http://example.com/pathtopdf
This will run at 3:00am every day.
This config is better for fcron :
%daily,mail(no) * * /usr/bin/wget --output-file="/pathtofolder/wget.log" --output-document="/pathtofolder/Document$(date +%H-%M-%Y-%m-%d).pdf" http://example.com/pathtopdf
Better alternative is make a script and put it in /etc/cron.daily/ and make it executable
#!/usr/bin/env bash
wget --output-file="/pathtofolder/wget.log" --output-document="/pathtofolder/Document$(date +%H-%M-%Y-%m-%d).pdf" http://example.com/pathtopdf
This will make files in specified folder like "Document20-36-2019-04-24.pdf"
answered 13 hours ago
LeonidMewLeonidMew
1,264624
Why the /dev/null at the end? So far I can not get it to work. May be the connection speed and I just have to wait...on mobile tethering.
– jau2019
12 hours ago
wget produces output while downloading a file. cron will send you mail with that output, or it can be redirected to /dev/null
– LeonidMew
12 hours ago
Remove "> /dev/null", run the script and see why it fails. You should edit folder path and url in script. Folder must have writeable permissions
– LeonidMew
12 hours ago
Worked when I ran it from terminal after removing /dev/null. I have fcron, rather than cron.
– jau2019
12 hours ago
I have add option to write log:--output-file="/pathtofolder/wget.log"so redirect to /dev/null no longer needed. If you dont want log, replace that option with--quiet
– LeonidMew
12 hours ago
|
show 1 more comment
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "89"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
jau2019 is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1137793%2fusing-wget-as-a-cron-job-to-fetch-contents-of-a-pdf-daily%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Edit cron jobs with crontab -e. Add this command:
0 3 * * * /usr/bin/wget --output-file="/pathtofolder/wget.log" --output-document="/pathtofolder/Document$(date +%H-%M-%Y-%m-%d).pdf" http://example.com/pathtopdf
This will run at 3:00am every day.
This config is better for fcron :
%daily,mail(no) * * /usr/bin/wget --output-file="/pathtofolder/wget.log" --output-document="/pathtofolder/Document$(date +%H-%M-%Y-%m-%d).pdf" http://example.com/pathtopdf
Better alternative is make a script and put it in /etc/cron.daily/ and make it executable
#!/usr/bin/env bash
wget --output-file="/pathtofolder/wget.log" --output-document="/pathtofolder/Document$(date +%H-%M-%Y-%m-%d).pdf" http://example.com/pathtopdf
This will make files in specified folder like "Document20-36-2019-04-24.pdf"
answered 13 hours ago
LeonidMewLeonidMew
1,264624
Why the /dev/null at the end? So far I can not get it to work. May be the connection speed and I just have to wait...on mobile tethering.
– jau2019
12 hours ago
wget produces output while downloading a file. cron will send you mail with that output, or it can be redirected to /dev/null
– LeonidMew
12 hours ago
Remove "> /dev/null", run the script and see why it fails. You should edit folder path and url in script. Folder must have writeable permissions
– LeonidMew
12 hours ago
Worked when I ran it from terminal after removing /dev/null. I have fcron, rather than cron.
– jau2019
12 hours ago
I have add option to write log:--output-file="/pathtofolder/wget.log"so redirect to /dev/null no longer needed. If you dont want log, replace that option with--quiet
– LeonidMew
12 hours ago
|
show 1 more comment
Edit cron jobs with crontab -e. Add this command:
0 3 * * * /usr/bin/wget --output-file="/pathtofolder/wget.log" --output-document="/pathtofolder/Document$(date +%H-%M-%Y-%m-%d).pdf" http://example.com/pathtopdf
This will run at 3:00am every day.
This config is better for fcron :
%daily,mail(no) * * /usr/bin/wget --output-file="/pathtofolder/wget.log" --output-document="/pathtofolder/Document$(date +%H-%M-%Y-%m-%d).pdf" http://example.com/pathtopdf
Better alternative is make a script and put it in /etc/cron.daily/ and make it executable
#!/usr/bin/env bash
wget --output-file="/pathtofolder/wget.log" --output-document="/pathtofolder/Document$(date +%H-%M-%Y-%m-%d).pdf" http://example.com/pathtopdf
This will make files in specified folder like "Document20-36-2019-04-24.pdf"
answered 13 hours ago
LeonidMewLeonidMew
1,264624
Why the /dev/null at the end? So far I can not get it to work. May be the connection speed and I just have to wait...on mobile tethering.
– jau2019
12 hours ago
wget produces output while downloading a file. cron will send you mail with that output, or it can be redirected to /dev/null
– LeonidMew
12 hours ago
Remove "> /dev/null", run the script and see why it fails. You should edit folder path and url in script. Folder must have writeable permissions
– LeonidMew
12 hours ago
Worked when I ran it from terminal after removing /dev/null. I have fcron, rather than cron.
– jau2019
12 hours ago
I have add option to write log:--output-file="/pathtofolder/wget.log"so redirect to /dev/null no longer needed. If you dont want log, replace that option with--quiet
– LeonidMew
12 hours ago
|
show 1 more comment
Edit cron jobs with crontab -e. Add this command:
0 3 * * * /usr/bin/wget --output-file="/pathtofolder/wget.log" --output-document="/pathtofolder/Document$(date +%H-%M-%Y-%m-%d).pdf" http://example.com/pathtopdf
This will run at 3:00am every day.
This config is better for fcron :
%daily,mail(no) * * /usr/bin/wget --output-file="/pathtofolder/wget.log" --output-document="/pathtofolder/Document$(date +%H-%M-%Y-%m-%d).pdf" http://example.com/pathtopdf
Better alternative is make a script and put it in /etc/cron.daily/ and make it executable
#!/usr/bin/env bash
wget --output-file="/pathtofolder/wget.log" --output-document="/pathtofolder/Document$(date +%H-%M-%Y-%m-%d).pdf" http://example.com/pathtopdf
This will make files in specified folder like "Document20-36-2019-04-24.pdf"
answered 13 hours ago
LeonidMewLeonidMew
1,264624
Edit cron jobs with crontab -e. Add this command:
0 3 * * * /usr/bin/wget --output-file="/pathtofolder/wget.log" --output-document="/pathtofolder/Document$(date +%H-%M-%Y-%m-%d).pdf" http://example.com/pathtopdf
This will run at 3:00am every day.
This config is better for fcron :
%daily,mail(no) * * /usr/bin/wget --output-file="/pathtofolder/wget.log" --output-document="/pathtofolder/Document$(date +%H-%M-%Y-%m-%d).pdf" http://example.com/pathtopdf
Better alternative is make a script and put it in /etc/cron.daily/ and make it executable
#!/usr/bin/env bash
wget --output-file="/pathtofolder/wget.log" --output-document="/pathtofolder/Document$(date +%H-%M-%Y-%m-%d).pdf" http://example.com/pathtopdf
This will make files in specified folder like "Document20-36-2019-04-24.pdf"
answered 13 hours ago
LeonidMewLeonidMew
1,264624
edited 12 hours ago
answered 13 hours ago
LeonidMewLeonidMew
1,264624
answered 13 hours ago
LeonidMewLeonidMew
1,264624
answered 13 hours ago
LeonidMewLeonidMew
1,264624
1,264624
Why the /dev/null at the end? So far I can not get it to work. May be the connection speed and I just have to wait...on mobile tethering.
– jau2019
12 hours ago
wget produces output while downloading a file. cron will send you mail with that output, or it can be redirected to /dev/null
– LeonidMew
12 hours ago
Remove "> /dev/null", run the script and see why it fails. You should edit folder path and url in script. Folder must have writeable permissions
– LeonidMew
12 hours ago
Worked when I ran it from terminal after removing /dev/null. I have fcron, rather than cron.
– jau2019
12 hours ago
I have add option to write log:--output-file="/pathtofolder/wget.log"so redirect to /dev/null no longer needed. If you dont want log, replace that option with--quiet
– LeonidMew
12 hours ago
|
show 1 more comment
Why the /dev/null at the end? So far I can not get it to work. May be the connection speed and I just have to wait...on mobile tethering.
– jau2019
12 hours ago
wget produces output while downloading a file. cron will send you mail with that output, or it can be redirected to /dev/null
– LeonidMew
12 hours ago
Remove "> /dev/null", run the script and see why it fails. You should edit folder path and url in script. Folder must have writeable permissions
– LeonidMew
12 hours ago
Worked when I ran it from terminal after removing /dev/null. I have fcron, rather than cron.
– jau2019
12 hours ago
I have add option to write log:--output-file="/pathtofolder/wget.log"so redirect to /dev/null no longer needed. If you dont want log, replace that option with--quiet
– LeonidMew
12 hours ago
Why the /dev/null at the end? So far I can not get it to work. May be the connection speed and I just have to wait...on mobile tethering.
– jau2019
12 hours ago
Why the /dev/null at the end? So far I can not get it to work. May be the connection speed and I just have to wait...on mobile tethering.
– jau2019
12 hours ago
wget produces output while downloading a file. cron will send you mail with that output, or it can be redirected to /dev/null
– LeonidMew
12 hours ago
wget produces output while downloading a file. cron will send you mail with that output, or it can be redirected to /dev/null
– LeonidMew
12 hours ago
Remove "> /dev/null", run the script and see why it fails. You should edit folder path and url in script. Folder must have writeable permissions
– LeonidMew
12 hours ago
Remove "> /dev/null", run the script and see why it fails. You should edit folder path and url in script. Folder must have writeable permissions
– LeonidMew
12 hours ago
Worked when I ran it from terminal after removing /dev/null. I have fcron, rather than cron.
– jau2019
12 hours ago
Worked when I ran it from terminal after removing /dev/null. I have fcron, rather than cron.
– jau2019
12 hours ago
I have add option to write log:
--output-file="/pathtofolder/wget.log" so redirect to /dev/null no longer needed. If you dont want log, replace that option with --quiet– LeonidMew
12 hours ago
I have add option to write log:
--output-file="/pathtofolder/wget.log" so redirect to /dev/null no longer needed. If you dont want log, replace that option with --quiet– LeonidMew
12 hours ago
|
show 1 more comment
jau2019 is a new contributor. Be nice, and check out our Code of Conduct.
jau2019 is a new contributor. Be nice, and check out our Code of Conduct.
jau2019 is a new contributor. Be nice, and check out our Code of Conduct.
jau2019 is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Ask Ubuntu!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1137793%2fusing-wget-as-a-cron-job-to-fetch-contents-of-a-pdf-daily%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Yea, I don't see this as being too hard with a little scripting. As to your question; I have a folder named DAILY where I would like to run this script from, using cron, to automatically: 1. Run Script. 2. Name downloaded file according to the days Date, NEVER overwriting the previous days VERSION. I want the pdfs to be searchable for research purposes. Naming the PDF file names is great, OR have wget/curl instead place each days download into a FOLDER according to the days date, would be fine.
– jau2019
13 hours ago
/user/site-xxx/DAILY
– jau2019
13 hours ago