如何通过在多个文本文件上使用 awk 并将摘要提取到一个文件中来提取字符串

Question

我有 70 个输入文件，文件名如 slurm-22801576.out、slurm-22801573.out、slurm-26801571.out 等。我想将所有需要的字符串提取到一个文件中。所以我做了以下操作，但我只能对一个文件这样做。如何在多个文件上执行此操作？

awk 'BEGIN{printf "file,reads,file,sample\n"}NR==32{printf "%s,%s,",FILENAME,}NR==2{printf "%s,%s,",FILENAME,}' slurm-22801576.out > summary/total_reads.csv

但是我的输出文件只有一行

file,reads,file,sample
slurm-22801576.out,outm=2006_42_aligned.sam,slurm-22801576.out,480789160,

在每个输入文件中，文本如下所示：

job starting at 23:42:13
java -ea -Xmx57039m -Xms57039m -cp /sw/bioinfo/bbmap/38.61b/rackham/current/ align2.BBMap build=1 overwrite=true fastareadlen=500 pairedonly=t ambiguous=toss secondary=t killbadpairs=t perfectmode=t minid=1 mappedonly=t outm=2006_40_aligned.sam scafstats=2006_40_fulllength.scafstats in=/crex/proj/datasets/human_depleted/Ki-2006-40-226_unmapped_R1.fq.gz in2=/crex/proj/datasets/human_depleted/Ki-2006-40-226_unmapped_R2.fq.gz threads=auto
Executing align2.BBMap [build=1, overwrite=true, fastareadlen=500, pairedonly=t, ambiguous=toss, secondary=t, killbadpairs=t, perfectmode=t, minid=1, mappedonly=t, outm=2006_40_aligned.sam, scafstats=2006_40_fulllength.scafstats, in=/crex/proj/datasets/human_depleted/Ki-2006-40-226_unmapped_R1.fq.gz, in2=/crex/proj/datasets/human_depleted/Ki-2006-40-226_unmapped_R2.fq.gz, threads=auto]
Version 38.61

Set OUTPUT_MAPPED_ONLY to true
Scaffold statistics will be written to 2006_40_fulllength.scafstats
Set threads to 10
Ambiguously mapped reads will be considered unmapped.
Set MINIMUM_ALIGNMENT_SCORE_RATIO to 1.000
Set genome to 1

Loaded Reference:       2.275 seconds.
Loading index for chunk 1-1, build 1
Generated Index:        4.174 seconds.
Analyzed Index:         3.394 seconds.
Started output stream:  0.264 seconds.
Creating scaffold statistics table:     0.064 seconds.
Cleared Memory:         1.216 seconds.
Processing reads in paired-ended mode.
Started read stream.
Started 10 mapping threads.
Detecting finished threads: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9

   ------------------   Results   ------------------

Genome:                 1
Key Length:             13
Max Indel:              0
Minimum Score Ratio:    1.0
Mapping Mode:           perfect
Reads Used:             461344228       (57756075474 bases)

Mapping:                595.769 seconds.
Reads/sec:              774367.86
kBases/sec:             96943.77

预期的输出文件应如下所示：

file,reads,file,sample
slurm-22801576.out,outm=2006_42_aligned.sam,slurm-22801576.out,480789160,
slurm-22801576.out,outm=2006_42_aligned.sam,slurm-22801576.out,480789160,
slurm-22801576.out,outm=2006_42_aligned.sam,slurm-22801576.out,480789160,
slurm-22801576.out,outm=2006_42_aligned.sam,slurm-22801576.out,480789160,

Answer 1

你可以使用这个awk:

awk '
   BEGIN{print "file,reads,file,sample"}
   FNR==2 {printf "%s,%s,", FILENAME, }
   FNR==32 {printf "%s,%s,\n", FILENAME, }
' slurm-*.out > summary/total_reads.csv

如何通过在多个文本文件上使用 awk 并将摘要提取到一个文件中来提取字符串

How to extract strings by using awk on multiple text files and summaries to one file

linux

awk