Run Data Pipelines Taskrunner jar forever on Existing Resources
When you run Data Pipelines Taskrunner
on your own resource, according to the following document , it do not exit out from shell.
To get the Taskrunner detached from the terminal, use nohup
, nohup
nohup java -jar /home/ec2-user/TaskRunner.jar —config /home/ec2-user/credentials.json --workerGroup=group-emr --region=us-west-2 --logUri=s3://mannem.ne/foldername &
An alternative is using screen/tmux/byobu
, which will keep the shell running, independent of the terminal.
Now, if you want to run this Taskrunner
every time your machine Boots
, It really depends on the OS distribution of your machine.
On a Amazon linux which are based on RedHat(RHEL)
:
Create a script like taskrunner-bootup
1 2 3 4 5 6 7 8 9 10 11 12 13 |
#!/bin/bash # chkconfig: 345 99 10 # says that the script must run at levels 3, 4 and 5 and the priority for start/stop is 99 and 10. # description: auto start Taskrunner # case "$1" in 'start') echo "Starting Taskrunner"; su - ec2-user -c "cd /home/ec2-user/ ; nohup java -jar /home/ec2-user/TaskRunner.jar —config /home/ec2-user/credentials.json --workerGroup=group-emr --region=us-west-2 --logUri=s3://logbucket.ne/foldername & ; 'stop') echo "Killing Taskrunner"; kill -9 `pgrep -f TaskRunner.jar`; esac |
Put your script in /etc/init.d/
, owned by root and executable. At the top of the script, you can give a directive for chkconfig. Example, the following script is used to start this Taskrunner java application as ec2-user. As user root you can use chkconfig
to enable or disable the script at startup,
chkconfig –list taskrunner-bootup
chkconfig –add taskrunner-bootup
and you can use service start/stop taskrunner-bootup
You can also use cloud-init
if you wish.
If this is an AMI, you can use USER-DATA script which only works during first start.
Here’s some solutions for
Ubuntu
: http://www.askubuntu.com/a/228313
Debian
: http://www.cyberciti.biz/tips/linux-how-to-run-a-command-when-boots-up.html