shell 脚本中的 psql 循环

Question

我需要从 file.csv 2 个值中读取并在 PostgreSQL 中进行超过 13.000 次查询。

正如您在上面的代码中看到的那样，这应该是一个非常简单的任务，但是存在一些问题。

#!/bin/bash


MSISDN=($(head file.csv | awk -F ";" '{print }' | sed -e "s/^/55/"))
APPID=($(head file.csv | awk -F ";" '{print }'))

NUMBER_OF_LINES=$(wc -l file.csv| grep -o "[0-9]*")

for i in $(seq 0 "$NUMBER_OF_LINES")
do

    export PGPASSWORD='MY_PASSWORD'
    psql -q -A -h VERY-LONG-HOST -U MYUSER -d DATABASE -p 1111 -t -c "select 'http://API-HOST/subscription/cancel?subscriptionId=' + s.subscription_id + '&phone=' + s.phone + '&enabled=0&statusId=7&notifyActionListeners=false&extraInfo=TICKET_NUMBER' from sbs.subscription s (nolock) join sbs.configuration c on s.configuration_id = c.configuration_id where c.application_id = ${APPID[$i]} and c.carrier_id = 2 and s.phone = ${MSISDN[$i]};"

done

代码执行时出现错误：

ERROR:  syntax error at or near "112940676229"
LINE 2: and c.carrier_id = 2 and s.phone = 55112940676229;

如何处理多个查询并在进行另一个查询之前关闭每个查询中的连接，到目前为止如何解决之前显示的错误？

file.csv中的内容示例：

112940676229;Sevice;333                                               
113429402012;Sevice;929  
111429402013;Sevice;888
11240672940;Sevice;445
11320294034;Sevice;333
11429294056;Sevice;22
11942940281;Sevice;122
11962940895;Sevice;233

Answer 1

为了安全和性能，我强烈建议为此使用不同的语言。也就是说，回答狭义的问题：

while IFS=';' read -r msidn _ appid _ <&3; do
  appid=${appid%$'\r'} # fix DOS newlines, should they be present in your input
  psql -q -A \
    -h VERY-LONG-HOST -U MYUSER -d DATABASE -p 1111 -t \
    -c "select 'http://API-HOST/subscription/cancel?subscriptionId=' + s.subscription_id + '&phone=' + s.phone + '&enabled=0&statusId=7&notifyActionListeners=false&extraInfo=TICKET_NUMBER' from sbs.subscription s (nolock) join sbs.configuration c on s.configuration_id = c.configuration_id where c.application_id = $appid and c.carrier_id = 2 and s.phone = $msidn;"
done 3< file.csv

也就是说，一个不那么糟糕的 Python 实现可能看起来像：

#!/usr/bin/env python

import csv
import sys
import psycopg2

query='''
select
  s.subscription_id,
  s.phone
from
  sbs.subscription s (nolock)
  join sbs.configuration c on s.configuration_id = c.configuration_id
where
  c.application_id = %(appid)s
  and c.carrier_id = 2
  and s.phone = %(msidn)s
'''

# user is expected to pass CSV file name as our first argument
filename = sys.argv[1]

# user is expected to pass a PostgreSQL connect string as our second argument
conn = psycopg2.connect(sys.argv[2])

curs = conn.cursor()

with open(filename, 'rb') as csv_file:
  csv_reader = csv.reader(csv_file, delimiter=';')
  for csv_row in csv_reader:
    msidn = csv_row[0]
    appid = csv_row[2]
    curs.execute(query, {"msidn": msidn, "appid": appid})
    for result_row in curs.fetchall():
      sid = result_row[0]
      phone = result_row[1]
      print 'http://API-HOST/subscription/cancel?subscriptionId=' + sid + '&phone=' + phone + '&enabled=0&statusId=7&notifyActionListeners=false&extraInfo=TICKET_NUMBER'

Answer 2

我建议您尝试这种方法：

create table file_csv (tel bigint, dk text, apid int);
copy file_csv from file.csv delimiter ';';

然后在您的查询中加入反对它。

更新感谢@Charles Duffy

可能会使它成为一个临时的 table (create temporary table file_csv (tel bigint, dk text, apid int))，如果意图是暂时的。

shell 脚本中的 psql 循环

psql loop in shell script

sql

postgresql

shell

for-loop

psql