解析csv文件并根据相对大小将行写入文件

Question

我有一长串不同时间的天气变量，我已经过滤掉那些不符合特定条件的变量。例如，所有数据点仅位于上午 11 点 (11) 和下午 5 点 (17pm) 之间。位于 11 点和 17 点之间的数据表示单个事件，并非每天都包含一个事件。我试图确定哪几天展示了一个事件。我知道一个新事件正在发生，每个时间 (HH24) 列中的值都低于它之前的值。例如。如果值 16（下午 5 点）后跟一个 11、12、13、14 或 15，那么我知道数据已经进入新的一天和事件。

我尝试编写的代码将获取包含最后一个值（例如 17）的行并将其写入文件，然后写入下一行。这样，新的 csv 文件将包含每个事件的开始时间（和其他信息）和结束时间。我假设我需要使用 for 命令，但我不确定如何使用 csv writer 来应对这个特殊的挑战。我的代码大纲如下所示，其中的部分我需要帮助遵循#

import csv

with open("weather_out_2000_2006_time_filtered_and_speed_filtered.csv", "rb") as input, open("X:\weatherresults\seabreezeevents.csv", "wb") as wanted:
    reader = csv.DictReader(input, delimiter=",", skipinitialspace=True)
    fieldnames = reader.fieldnames
    writer_wanted = csv.DictWriter(wanted, fieldnames, delimiter=",")
    writer_wanted.writeheader()

    for line_number, row in enumerate(reader):
        try:
            if float(row["HH24"]) < #the value in the subsequent row:
                writer_wanted.writerow(row) # and also write subsequent row
        except:
            print "Failed to parse line", line_number
            print row

我的数据文件如下所示。我已经展示了 HH24 从高值到低值的转变，所以你可以明白我的意思。

hd,Station Number,Year Month Day Hours Minutes in YYYY,MM,DD,HH24,MI format in Local time,Year Month Day Hours Minutes in YYYY,MM,DD,HH24,MI format in Local standard time,Year Month Day Hours Minutes in YYYY,MM,DD,HH24,MI format in Universal coordinated time,Precipitation since last (AWS) observation in mm,Quality of precipitation since last (AWS) observation value,Air Temperature in degrees Celsius,Quality of air temperature,Air temperature (1-minute maximum) in degrees Celsius,Quality of air temperature (1-minute maximum),Air temperature (1-minute minimum) in degrees Celsius,Quality of air temperature (1-minute minimum),Wet bulb temperature in degrees Celsius,Quality of Wet bulb temperature,Wet bulb temperature (1 minute maximum) in degrees Celsius,Quality of wet bulb temperature (1 minute maximum),Wet bulb temperature (1 minute minimum) in degrees Celsius,Quality of wet bulb temperature (1 minute minimum),Dew point temperature in degrees Celsius,Quality of dew point temperature,Dew point temperature (1-minute maximum) in degrees Celsius,Quality of Dew point Temperature (1-minute maximum),Dew point temperature (1 minute minimum) in degrees Celsius,Quality of Dew point Temperature (1 minute minimum),Relative humidity in percentage %,Quality of relative humidity,Relative humidity (1 minute maximum) in percentage %,Quality of relative humidity (1 minute maximum),Relative humidity (1 minute minimum) in percentage %,Quality of Relative humidity (1 minute minimum),Wind (1 minute) speed in km/h,Wind (1 minute) speed quality,Minimum wind speed (over 1 minute) in km/h,Minimum wind speed (over 1 minute) quality,Wind (1 minute) direction in degrees true,Wind (1 minute) direction quality,Standard deviation of wind (1 minute),Standard deviation of wind (1 minute) direction quality,Maximum wind gust (over 1 minute) in km/h,Maximum wind gust (over 1 minute) quality,Visibility (automatic - one minute data) in km,Quality of visibility (automatic - one minute data),Mean sea level pressure in hPa,Quality of mean sea level pressure,Station level pressure in hPa,Quality of station level pressure,QNH pressure in hPa,Quality of QNH pressure,#
    hd,40842,2000,3,22,13,40,2000,3,22,13,40,2000,3,22,13,40,0,N,20.4,N,20.5,N,20.4,N,20.2,N,20.2,N,20.1,N,20.1,N,20.1,N,20,N,98,N,,N,,N,9,N,8,N,18,N,7,N,11,N,,N,1013.3,N,1012.2,N,1013.3,N,#
    hd,40842,2000,3,22,13,47,2000,3,22,13,47,2000,3,22,13,47,0,N,20.5,N,20.5,N,20.5,N,20.2,N,20.2,N,20.2,N,20.1,N,20.1,N,20,N,97,N,,N,,N,4,N,0,N,56,N,75,N,5,N,,N,1013.2,N,1012.1,N,1013.2,N,#
    hd,40842,2000,3,23,11,0,2000,3,23,11,0,2000,3,23,11,0,0,N,23.4,N,23.4,N,23.3,N,21.3,N,21.4,N,21.3,N,20.2,N,20.3,N,20.2,N,82,N,,N,,N,8,N,5,N,66,N,2,N,9,N,,N,1013.6,N,1012.5,N,1013.6,N,#
    hd,40842,2000,3,23,11,1,2000,3,23,11,1,2000,3,23,11,1,0,N,23.4,N,23.4,N,23.4,N,21.4,N,21.4,N,21.3,N,20.3,N,20.3,N,20.2,N,82,N,,N,,N,8,N,5,N,68,N,3,N,9,N,,N,1013.6,N,1012.5,N,1013.6,N,#

Answer 1

您想在日期更改时写一行，所以我认为最好创建一个日期变量进行比较。（我在评论中提到为什么仅比较 "HH24" 值无法确定您何时到达新日期）

跟踪和写出前行（因为您已经处理过它）比后续行要容易得多行，所以这就是你应该如何考虑继续。像下面这样的东西应该有所帮助（未经测试）：

...
import datetime
prev_row = None
for line_number, row in enumerate(reader):
    try:
        dt = datetime.date(year=row["Year"], month=row["Month"], day=row["day"])
        if prev_row is not None and dt > prev_row['dt']:
            writer_wanted.writerow(prev_row['row'])
            writer_wanted.writerow(row)
        prev_row = {'row':row, 'dt':dt}
    except:
        print "Failed to parse line", line_number
        print row

编辑：

您的程序中这一行的第一部分：

with open("weather_out_2000_2006_time_filtered_and_speed_filtered.csv", "rb") as input

打开命名的 .csv 文件进行输入（因为 mode 是 'rb' - docs here）。

同一行的下一部分：

open("X:\weatherresults\seabreezeevents.csv", "wb") as wanted

打开该命名文件进行输出（'wb' 模式）- 请参阅与上述相同的参考。

此时，变量名 input 和 wanted 现在都引用 file type.

的对象

您的程序接下来使用 csv 模块以特定方式读取文件，这有助于解析以逗号分隔的文本文件；并将该引用分配给 reader 变量。同样，它将变量 writer_wanted 分配给 csv.DictWriter，这将有助于在将行写入输出文件时进行格式化，由 wanted.

引用

之后，一次一行读入：

for line_number, row in enumerate(reader):

一次写一行：

writer_wanted.writerow(row)

如果您想要更多详细信息，最好的办法是阅读一些 Python 教程（Google 是您的朋友）。

解析csv文件并根据相对大小将行写入文件

Parsing csv file and writing rows to file according to relative size

python

csv

parsing

python-2.7

export-to-csv