Flask 服务器事件在开发中工作但不在生产中
Flask server events working in development but not in production
我有一个应该向浏览器生成服务器事件的烧瓶路由。基本上该功能的作用是:
1.加载一个csv文件
2.对于csv文件的每一行
3. 将用户名和电子邮件保存在 sql 数据库中(使用 sqlalchemy)
4.更新计数器(进度状态)
5. 向浏览器发送事件
问题是当我处于开发模式(使用 flask 内置服务器)时该功能运行良好,但在生产模式(使用 NginX 和 gunicorn)时该功能在几秒钟后停止,因此计数器永远不会达到 100,它会导致浏览器再次调用该函数,并且此循环永远不会结束,因为事件永远不会得到关闭语句。所以主要问题是,为什么它适用于开发而不是生产?
这是mi代码:
# Update or construct database if a csv file was submitted
@app.route("/constructDatabase/<string:filename>", methods=["GET","POST"])
def constructDatabase(filename):
# context manager to open csv file
csvFile = open(os.path.join(os.getcwd(), filename), newline='')
# get lines count of csv file
totalLines = len(csvFile.readlines())
# reset reader pointer
csvFile.seek(0)
# current percent status
current_status = 0
def generate(file, counter):
# unpack and iterate over the csv file to get all the names and emails
for Company,Address,City,State,Zip,County,Phone,Website,Contact,Title,Direct_Phone,Email,Sales,Employees,SIC_Code,Industry in csv.reader(file, delimiter=','):
yield ':keep connection alive\n\n'
counter += 1
# if a user has not contact name or email then not useful
if Email == None or Email == '':
yield f'id: {counter}\nevent: message\ndata: {round(counter / totalLines * 100, 1)}\n\n'
continue
if Contact == None or Contact == '':
yield f'id: {counter}\nevent: message\ndata: {round(counter / totalLines * 100, 1)}\n\n'
continue
# Create user as instance of User class
user = Users(company=Company, address=Address, city=City, state=State,
zip=Zip, country=County, phone=Phone, website=Website, contact=Contact,
title=Title, direct_phone = Direct_Phone, email=Email, sales=Sales,
employees=Employees, sic_code=SIC_Code, industry=Industry)
# Add user to database
db.session.add(user)
# get current percent status of building database
yield f'id: {counter}\nevent: message\ndata: {round(counter / totalLines * 100, 1)}\n\n'
# Save changes in database
db.session.commit()
print("SAVING DATABASE .......")
# close file
file.close()
return Response(generate(csvFile, current_status), mimetype='text/event-stream')
Java 现在的脚本代码:
javascript
// create Event source connection with the server to listen for incoming msg
var source = new EventSource(`/constructDatabase/${filename}`);
// if new msg was received
source.onmessage = function(msg) {
// update progress bar
$('.progress-bar').css('width', msg.data+'%').attr('aria-valuenow', msg.data);
// if is 100 percent close connection to the server
if (msg.data == 100) {
source.close();
// Hide label
$('.prog-bar-label').addClass('d-none');
// Hide CSV progress bar
$('.csvProgressBar').addClass('d-none');
// reset progress bar
$('.csvProgressBar').find('.progress-bar').css('width', 0+'%').attr('aria-valuenow', 0);
}
};
source.onerror = function(error){
console.log(error.data);
};
我明白这是怎么回事了。问题是 flask 开发人员服务器在运行时没有声明任何超时,因此一个函数可能会等待很长时间的响应。但是在带有gunicorn的生产服务器中,它有一个默认的超时时间很短,所以如果一个函数需要很长时间才能得到响应并且达到超时时间,gunicorn会杀死进程并退出该函数,所以浏览器端的流进程,认为那是服务器中的错误,当时只是一个延迟的功能。解决方案是使用 --timeout(此处为所需的超时值)增加 gunicorn 中的超时,这会很好。不要使用非常大的超时值,只需尝试弄清楚您的函数需要多长时间才能获得响应并使用该值。请记住,此值现在对于您的所有函数都是相同的。
我有一个应该向浏览器生成服务器事件的烧瓶路由。基本上该功能的作用是: 1.加载一个csv文件 2.对于csv文件的每一行 3. 将用户名和电子邮件保存在 sql 数据库中(使用 sqlalchemy) 4.更新计数器(进度状态) 5. 向浏览器发送事件
问题是当我处于开发模式(使用 flask 内置服务器)时该功能运行良好,但在生产模式(使用 NginX 和 gunicorn)时该功能在几秒钟后停止,因此计数器永远不会达到 100,它会导致浏览器再次调用该函数,并且此循环永远不会结束,因为事件永远不会得到关闭语句。所以主要问题是,为什么它适用于开发而不是生产? 这是mi代码:
# Update or construct database if a csv file was submitted
@app.route("/constructDatabase/<string:filename>", methods=["GET","POST"])
def constructDatabase(filename):
# context manager to open csv file
csvFile = open(os.path.join(os.getcwd(), filename), newline='')
# get lines count of csv file
totalLines = len(csvFile.readlines())
# reset reader pointer
csvFile.seek(0)
# current percent status
current_status = 0
def generate(file, counter):
# unpack and iterate over the csv file to get all the names and emails
for Company,Address,City,State,Zip,County,Phone,Website,Contact,Title,Direct_Phone,Email,Sales,Employees,SIC_Code,Industry in csv.reader(file, delimiter=','):
yield ':keep connection alive\n\n'
counter += 1
# if a user has not contact name or email then not useful
if Email == None or Email == '':
yield f'id: {counter}\nevent: message\ndata: {round(counter / totalLines * 100, 1)}\n\n'
continue
if Contact == None or Contact == '':
yield f'id: {counter}\nevent: message\ndata: {round(counter / totalLines * 100, 1)}\n\n'
continue
# Create user as instance of User class
user = Users(company=Company, address=Address, city=City, state=State,
zip=Zip, country=County, phone=Phone, website=Website, contact=Contact,
title=Title, direct_phone = Direct_Phone, email=Email, sales=Sales,
employees=Employees, sic_code=SIC_Code, industry=Industry)
# Add user to database
db.session.add(user)
# get current percent status of building database
yield f'id: {counter}\nevent: message\ndata: {round(counter / totalLines * 100, 1)}\n\n'
# Save changes in database
db.session.commit()
print("SAVING DATABASE .......")
# close file
file.close()
return Response(generate(csvFile, current_status), mimetype='text/event-stream')
Java 现在的脚本代码:
javascript
// create Event source connection with the server to listen for incoming msg
var source = new EventSource(`/constructDatabase/${filename}`);
// if new msg was received
source.onmessage = function(msg) {
// update progress bar
$('.progress-bar').css('width', msg.data+'%').attr('aria-valuenow', msg.data);
// if is 100 percent close connection to the server
if (msg.data == 100) {
source.close();
// Hide label
$('.prog-bar-label').addClass('d-none');
// Hide CSV progress bar
$('.csvProgressBar').addClass('d-none');
// reset progress bar
$('.csvProgressBar').find('.progress-bar').css('width', 0+'%').attr('aria-valuenow', 0);
}
};
source.onerror = function(error){
console.log(error.data);
};
我明白这是怎么回事了。问题是 flask 开发人员服务器在运行时没有声明任何超时,因此一个函数可能会等待很长时间的响应。但是在带有gunicorn的生产服务器中,它有一个默认的超时时间很短,所以如果一个函数需要很长时间才能得到响应并且达到超时时间,gunicorn会杀死进程并退出该函数,所以浏览器端的流进程,认为那是服务器中的错误,当时只是一个延迟的功能。解决方案是使用 --timeout(此处为所需的超时值)增加 gunicorn 中的超时,这会很好。不要使用非常大的超时值,只需尝试弄清楚您的函数需要多长时间才能获得响应并使用该值。请记住,此值现在对于您的所有函数都是相同的。