使用 Unix shell 脚本维护检查点

Question

以下代码在管理检查点时效率不高。有人能给出更好的主意吗？

MainScriptName="MainScriptName-"
Checkpoints="$MAIN/Checkpoints"
StepNo="02"
StepName="MyStepName"
MainLogPathFilename="$LogPath$MainScriptName$LogFilenamets.csv"

fileCount=0
fileCount=`find $Checkpoints -maxdepth 1 -name "$MainScriptName$StepNo.lck" | wc -l`
SECONDS=0
executions_status=""
StepStarted="$(date +'%d/%m/%Y %H:%M')"
echo "Step No : $StepNo $StepStarted"
if [ $fileCount -eq 0 ] 
then

    $hivef $MAIN/MyscriptName/Date.hql  >> $LogPath/$MainScriptName.log 2>&1 

    if [ "$?" -ne 0 ] 
    then 
        duration=$SECONDS
        echo "$StepName, $StepStarted, $StepCompleted, $(($duration / 60)), Failed" >> $MainLogPathFilename
        echo "Failed !"
        exit 10
    else 
        touch $Checkpoints/$MainScriptName$StepNo.lck
        executions_status="Success"

    fi


else
    executions_status="Skipped"

fi
    echo "$executions_status"
    StepCompleted="$(date +'%d/%m/%Y %H:%M')"
    duration=$SECONDS
    echo "$StepNo, $StepName, $StepStarted, $StepCompleted, $(($duration / 60)), $executions_status" >> $MainLogPathFilename

然后简单地阅读它（如果存在）然后跳过它。

Answer 1

echo "Script Name : " $MainScriptName "- Started"
StepNo="01"
fileCount=0
fileCount=`find $Checkpoints -maxdepth 1 -name "$MainScriptName$StepNo.lck" | wc -l`
StepName="Build_Report"
SECONDS=0
executions_status=""
StepStarted="$(date +'%d/%m/%Y %H:%M')"

echo "File Count : $fileCount"
echo "Step No : $StepNo $StepStarted"
if [ $fileCount -eq 0 ] then

    $hivef $HDP_MAIN/CODE/TXN/HiveQuery.hql > $LogPath/$MainScriptName.log 2>&1
    query1result=$?
    ScriptError="-1" 
        ScriptError=`cat "$LogPath/$MainScriptName.log" | grep "(No such file or directory)\|Error opening session" | wc -l` 
    if [ $query1result -ne 0 ] || [ $ScriptError -gt 0 ]
    then 
        duration=$SECONDS
        echo "$StepName, $StepStarted, $StepCompleted, $(($duration / 60)), Failed" >> $MainLogPathFilename
        echo "Failed !"
        exit 10
    else 
        touch $Checkpoints/$MainScriptName$StepNo.lck
        executions_status="Successfully Completed."

    fi
else
    executions_status="Skipped"

fi
    echo "$executions_status"
    StepCompleted="$(date +'%d/%m/%Y %H:%M')"
    duration=$SECONDS
    echo "$StepNo, $StepName, $StepStarted, $StepCompleted, $(($duration / 60)), $executions_status" >> $MainLogPathFilename

使用 Unix shell 脚本维护检查点

maintaining checkpoints using Unix shell script

unix

hive