如何删除文本文件中的重复链接？

Question

所以我有一个文本文件，里面看起来像这样：

http://example.pl/folder/this_same1.avi
http://example.pl/folder/this_same1.avi
http://example.pl/folder/this_same2.avi
http://example.pl/folder/this_same2.avi
http://example.pl/folder/this_same3.avi
http://example.pl/folder/this_same3.avi

我想删除所有重复的链接。 输出文件如下所示：

http://example.pl/folder/this_same1.avi
http://example.pl/folder/this_same2.avi
http://example.pl/folder/this_same3.avi

Answer 1

哦，我改进了我的答案：

links = set()
with open('file.txt', 'r') as fp:
    for line in fp.readlines():
        links.add(line)

然后就可以回写文件了：

with open('file.txt', 'wb') as fp:
    for line in links:
        fp.write(line)

自己测试..

Answer 2

如果结构一致且顺序很重要：

links = fp.readlines()[::2]

如果结构不一致，顺序很重要：

links = []
for line in fp.readlines():
    if line not in links:
        links.append(line)

然后写入文件。