Boss: “Can you check out how much time it’s taking our workflows to complete? Me: “Sure thing!” googles oozie docs Co-worker: “Oh my gosh, I’ve just been looking at the timestamps and doing the calculation manually! It took me all weekend!”

Here’s some things that might help you to be lazy like me!

Oozie is Apache’s job manager. You can start/stop/resume/suspend/rerun workflows - a bundle of one or more Hadoop Map/Reduce and Pig jobs. Oozie CLI has some AWESOME documentation which you should totally leverage so that you can go home on the weekends instead of manually checking all of your jobs in a slow web app - looks sideways at Ambari.

Thing 1: oozie job returns you a list of the most recently executed workflow jobs

Thing 2: You can limit the results you get with the -len parameter

Thing 3: You can filter the types of jobs with the -filter parameter. You can filter on several aspects of a job:

• name
• user
• group
• status
• frequency
• unit

And oozie even allows you to use multiple filters -filter name=wf_name;user=euler

Thing 4: Combine oozie’s output with a little grep action to extend your filtering (or hide all of the ascii table stuff you get back from oozie)

Thing 5: The “how long did this take” part is pure bash script. You can take advantage of the date method in order to format your timestamps or do timestamp math.

Here’s an example of how I find all of the succeeded jobs’ duration.

#!/bin/sh

nJobs=$1
searchCriteria=$2

oozie jobs -localtime -len $nJobs -filter status='SUCCEEDED' | grep $searchCriteria | while read -r line; do job=($line);
start=$(date "+%s" -d "${job[4]} ${job[5]} ${job[6]}"); end=$(date "+%s" -d "${job[7]} ${job[8]} ${job[9]}");
echo ${job[1]} $((($end - $start)/60)) " mins";done | sed 's/SUCCEEDED//'