Python恢复两个列表的中断迭代
Python restoring an interrupt iteration of two lists
这里是代码:
import itertools
# import list from file
a = [line.strip() for line in open("real.txt", 'r')]
b = [line.strip() for line in open("imag.txt", 'r')]
# create file contain restore informations
f = open("restore.txt","w+")
loop = 0
try:
for r in itertools.product(a,b):
loop +=1
print "processing : "+ r[1] + ": "+ r[0] + " ("+str(loop)+")"
except Exception, e:
print str(e)
f.write("iterazions seccession: " +str(loop)+"\n")
f.write("real number : " +r[1]+"\n")
f.write("imaginary unit: " + r[0]+ "\n")
示例输出
processing: 1 : 1i (1)
processing: 1 : 2i (2)
processing: 1 : 3i (3)
processing: 1 : 4i (4)
...
processing: 2000 : 174i (348000)
processing: 2000 : 175i (348001)
...and so forth
(it does all combinations of two lists)
问题是如果错误停止迭代,有没有办法从上次迭代重新启动脚本而不从头开始?
我尝试将最后一个值保存在文件中,但我不知道如何使用它。
p.s。我知道复数的 cmath 函数,但我更感兴趣的是恢复问题
更具体
如果错误停止迭代:
处理:2000 : 175i (348001)
有一种方法可以从 2000 重新启动脚本:175i (348001) iteration?
将 try/except 放在 for 循环中 - 如果 except 没有引发,则 for 循环将继续。
for r in itertools.product(a,b):
try:
loop +=1
print "processing : "+ r[1] + ": "+ r[0] + " ("+str(loop)+")"
#enable for testing error
#if loop == 15:
#loop = loop/0
except Exception, e:
print str(e)
f.write("iterazions seccession: " +str(loop)+"\n")
f.write("real number : " +r[1]+"\n")
f.write("imaginary unit: " + r[0]+ "\n")
通常您会看到 try ... except
块的使用,这些块在 Python 中相对便宜。
这不会停止循环的执行,如本例所示(可适用于您的情况):
numbers = [1, 3, 2, 5, 0, 4, 5]
for number in numbers:
try:
print(10/number)
except ZeroDivisionError:
print("Please don't divide by 0...")
您将在 Python 2.7.11 中看到此输出是:
10
3
5
2
Please don't divide by 0...
2
2
如果您的代码可以抛出多种类型的异常,您可以将它们链接起来:
numbers = [1, 3, 2, 5, 0, 4, 5]
for number in numbers:
try:
print(10/number)
except ZeroDivisionError:
print("Please don't divide by 0...")
except ValueError:
print("Something something ValueError")
except KeyError:
print("Something something KeyError")
如果您知道您的代码可能引发的所有异常类型,您就可以适当地处理它们,这通常是比异常的一般解决方案更好的解决方案。
关于您的代码的一般说明:
为变量使用更具代表性的名称
- 使用更具代表性的名称将有助于 user/maintainer 了解您的代码的情况 并且 可以帮助您发现错误。
a
和 b
在实数和虚数方面意义不大 - 为什么不使用 real_nums
和 imag_nums
?没有更多的打字和更多的描述。
使用open(file) as f
- 这将自动关闭您的文件,因此您以后无需担心,这对 writing/appending
来说可能是个问题
使用enumerate(iterable)
- 这将跟踪迭代次数和迭代时的对象
遍历元组
- 您不需要使用
[]
符号访问 - 您可以直接使用元组中的元素。
使用格式字符串
- 这些将使您的代码在打印时看起来更漂亮
将所有这些放在一起:
import itertools
# import list from file
reals = [line.strip() for line in open("real.txt", 'r')]
imags = [line.strip() for line in open("imag.txt", 'r')]
# create file contain restore informations
with open("restore.txt", "w+") as f:
for i, (real, imag) in enumerate(itertools.product(reals, imags)):
try:
print "processing: {0}, {1}, ({2})".format(real, imag, i)
# Process data here
#enable for testing error
if i == 15:
i = i/0
except ZeroDivisionError:
# Handle exception here
continue
except Exception as e:
print str(e)
f.write("iterazions seccession: {0}\n".format(i))
f.write("real number: {0}\n".format(real))
f.write("imaginary unit: {0}\n".format(imag))
我的解决方案
所以,我的问题是 "how restart the script from last iteration without start from scratch?"。
首先,我在 hard way 中尝试,即使它工作正常,我也意识到一种比那简单得多的方法(亲吻吧?)。这里的代码:
import itertools
import pickle #for restoring last data session
fname = "restore.pck"
reals = [line.strip() for line in open("real.txt", 'r')]
imags = [line.strip() for line in open("imag.txt", 'r')]
seed_count = 0 #number last processing cycle
count_ses = 0 #number current session cycle
count_tot = 0 #number total cycle processing so far
it_tot=len(reals)*len(imgs)
ret = raw_input("recover last session?: y/n")
if ret =="y":
with open(fname,"rb") as f:
rec= pickle.load(f)
print "last session data {0} {1} {2}".format(rec[0],rec[1],rec[2])
seed_count=rec[2]
seed_img=rec[1]
seed_rea=rec[0]
try:
print "total iterations", it_tot
for r in itertools.product(reals,imgs):
# do nothing until you get last interrupted cycle
count_tot+=1
if count_tot >= seed_count:
# and now start processing
count_ses +=1
print "processing: {0}, {1}, (total : {2}), (this session: {3}) {4}% )".format(r[0], r[1], count_tot, count_ses,(count_tot/len_tot*100))
except Exception as e:
print str(e)
#save data if an error occurred
store = r[0], r[1], count_tot
with open(fname,"wb") as f:
pickle.dump(store,f,pickle.HIGHEST_PROTOCOL)
f.close()
只是让运行迭代,不做任何处理,直到到达最后一个处理周期,然后重新开始计算。
当然有些"wiseguy"会说这不是pythonic方式或者这个代码一团糟,不像我的那样花哨 但这是我的第一个 python 程序,它有效而且我很高兴:)
这里是代码:
import itertools
# import list from file
a = [line.strip() for line in open("real.txt", 'r')]
b = [line.strip() for line in open("imag.txt", 'r')]
# create file contain restore informations
f = open("restore.txt","w+")
loop = 0
try:
for r in itertools.product(a,b):
loop +=1
print "processing : "+ r[1] + ": "+ r[0] + " ("+str(loop)+")"
except Exception, e:
print str(e)
f.write("iterazions seccession: " +str(loop)+"\n")
f.write("real number : " +r[1]+"\n")
f.write("imaginary unit: " + r[0]+ "\n")
示例输出
processing: 1 : 1i (1)
processing: 1 : 2i (2)
processing: 1 : 3i (3)
processing: 1 : 4i (4)
...
processing: 2000 : 174i (348000)
processing: 2000 : 175i (348001)
...and so forth
(it does all combinations of two lists)
问题是如果错误停止迭代,有没有办法从上次迭代重新启动脚本而不从头开始? 我尝试将最后一个值保存在文件中,但我不知道如何使用它。
p.s。我知道复数的 cmath 函数,但我更感兴趣的是恢复问题
更具体
如果错误停止迭代: 处理:2000 : 175i (348001) 有一种方法可以从 2000 重新启动脚本:175i (348001) iteration?
将 try/except 放在 for 循环中 - 如果 except 没有引发,则 for 循环将继续。
for r in itertools.product(a,b):
try:
loop +=1
print "processing : "+ r[1] + ": "+ r[0] + " ("+str(loop)+")"
#enable for testing error
#if loop == 15:
#loop = loop/0
except Exception, e:
print str(e)
f.write("iterazions seccession: " +str(loop)+"\n")
f.write("real number : " +r[1]+"\n")
f.write("imaginary unit: " + r[0]+ "\n")
通常您会看到 try ... except
块的使用,这些块在 Python 中相对便宜。
这不会停止循环的执行,如本例所示(可适用于您的情况):
numbers = [1, 3, 2, 5, 0, 4, 5]
for number in numbers:
try:
print(10/number)
except ZeroDivisionError:
print("Please don't divide by 0...")
您将在 Python 2.7.11 中看到此输出是:
10
3
5
2
Please don't divide by 0...
2
2
如果您的代码可以抛出多种类型的异常,您可以将它们链接起来:
numbers = [1, 3, 2, 5, 0, 4, 5]
for number in numbers:
try:
print(10/number)
except ZeroDivisionError:
print("Please don't divide by 0...")
except ValueError:
print("Something something ValueError")
except KeyError:
print("Something something KeyError")
如果您知道您的代码可能引发的所有异常类型,您就可以适当地处理它们,这通常是比异常的一般解决方案更好的解决方案。
关于您的代码的一般说明:
为变量使用更具代表性的名称
- 使用更具代表性的名称将有助于 user/maintainer 了解您的代码的情况 并且 可以帮助您发现错误。
a
和b
在实数和虚数方面意义不大 - 为什么不使用real_nums
和imag_nums
?没有更多的打字和更多的描述。
使用open(file) as f
- 这将自动关闭您的文件,因此您以后无需担心,这对 writing/appending 来说可能是个问题
使用enumerate(iterable)
- 这将跟踪迭代次数和迭代时的对象
遍历元组
- 您不需要使用
[]
符号访问 - 您可以直接使用元组中的元素。
使用格式字符串
- 这些将使您的代码在打印时看起来更漂亮
将所有这些放在一起:
import itertools
# import list from file
reals = [line.strip() for line in open("real.txt", 'r')]
imags = [line.strip() for line in open("imag.txt", 'r')]
# create file contain restore informations
with open("restore.txt", "w+") as f:
for i, (real, imag) in enumerate(itertools.product(reals, imags)):
try:
print "processing: {0}, {1}, ({2})".format(real, imag, i)
# Process data here
#enable for testing error
if i == 15:
i = i/0
except ZeroDivisionError:
# Handle exception here
continue
except Exception as e:
print str(e)
f.write("iterazions seccession: {0}\n".format(i))
f.write("real number: {0}\n".format(real))
f.write("imaginary unit: {0}\n".format(imag))
我的解决方案
所以,我的问题是 "how restart the script from last iteration without start from scratch?"。 首先,我在 hard way 中尝试,即使它工作正常,我也意识到一种比那简单得多的方法(亲吻吧?)。这里的代码:
import itertools
import pickle #for restoring last data session
fname = "restore.pck"
reals = [line.strip() for line in open("real.txt", 'r')]
imags = [line.strip() for line in open("imag.txt", 'r')]
seed_count = 0 #number last processing cycle
count_ses = 0 #number current session cycle
count_tot = 0 #number total cycle processing so far
it_tot=len(reals)*len(imgs)
ret = raw_input("recover last session?: y/n")
if ret =="y":
with open(fname,"rb") as f:
rec= pickle.load(f)
print "last session data {0} {1} {2}".format(rec[0],rec[1],rec[2])
seed_count=rec[2]
seed_img=rec[1]
seed_rea=rec[0]
try:
print "total iterations", it_tot
for r in itertools.product(reals,imgs):
# do nothing until you get last interrupted cycle
count_tot+=1
if count_tot >= seed_count:
# and now start processing
count_ses +=1
print "processing: {0}, {1}, (total : {2}), (this session: {3}) {4}% )".format(r[0], r[1], count_tot, count_ses,(count_tot/len_tot*100))
except Exception as e:
print str(e)
#save data if an error occurred
store = r[0], r[1], count_tot
with open(fname,"wb") as f:
pickle.dump(store,f,pickle.HIGHEST_PROTOCOL)
f.close()
只是让运行迭代,不做任何处理,直到到达最后一个处理周期,然后重新开始计算。 当然有些"wiseguy"会说这不是pythonic方式或者这个代码一团糟,不像我的那样花哨 但这是我的第一个 python 程序,它有效而且我很高兴:)