Ruby 测试脚本使用 curl 命令将文件上传到 gcs 失败

Question

我在通过签名 url 上传到 Google 云存储时遇到问题。

为了找出问题所在，在 ruby 中写了一个小测试脚本。

#
#
#     Script to see how upload ror url generation works.
#
#

require "google/cloud/storage"
require 'digest'

project_id = "Your project_id"
bucket_name = "your_bucket"

# commandline...
unless ARGV.length == 1 or ARGV.length == 2
  puts "Usage: gcs_upload_file.rb file_to_upload content_type (default image/jpeg) \n"
  exit
end

@Content_type = "image/jpeg"
file_path = ARGV[0]

if !ARGV[1].nil?
  @Content_type = ARGV[1]
end

# File check
if !File.exist? file_path
  puts "Unable to open file #{ARGV[0]}\n"
  exit
end

def hex_to_base64_digest(hexdigest)
  [[hexdigest].pack("H*")].pack("m").strip
end

# make md5 and file size
file_size = File.size file_path
digest = Digest::MD5.file(file_path).hexdigest
Content_md5 = hex_to_base64_digest digest

p file_path
p "file_size = #{file_size}"
p "digest = #{digest}"
p "Content_md5 =  #{Content_md5}"

gcloud = Google::Cloud.new project_id, "./config/gcs.json"
storage = gcloud.storage
bucket = storage.bucket bucket_name

p "So far so good"
#p bucket.inspect
headers = Hash.new

# bug bug bug
# this should be changed into:
# sign = Google::Cloud::Storage::File::SignerV2.from_bucket bucket, file_path
sign = Google::Cloud::Storage::File::SignerV2.from_bucket bucket, nil
generated_url = sign.signed_url method: "PUT",
                                expires: 120,
                                headers: headers,
                                content_type: @Content_type,
                                content_md5: Content_md5,
                                issuer: nil,
                                client_email: nil,
                                signing_key: nil,
                                private_key: nil,
                                signer: nil,
                                query: nil

p "Generated url:"
p generated_url



commando = "curl -X PUT \"" + generated_url + '"'
commando += " -H \"Content-Type: #{@Content_type}\" "
#commando += " -H Content-length:4254548 "
commando += " -H Content-length:#{file_size} "
#commando += " -H \"Content-MD5: 4IXaimkAX9KasFuXznB3jg==\" "
commando += " -H \"Content-MD5: #{Content_md5}\" "
commando += " --upload-file #{file_path} "
# p 'curl -X PUT "#{generated_url}" -H "Content-Type: image/jpeg" -H "Content-length: 100" --upload-file test.jpg '
#exec('curl -X PUT "#{generated_url}" -H "Content-Type: image/jpeg" -H "Content-length: 100" --upload-file test.jpg ')
p commando
exec(commando)

p "End of story !"

我觉得 url 是正确的，因为在其他情况下我收到的签名消息不正确。

我发送一个文本文件hello_world.txt:

Start
Hello World
Hello World
Hello World
End

命令行是：

ruby gcs_upload_file.rb hello_world.txt plain/text

但是上传失败，脚本输出为：

"hello_world.txt"
"file_size = 51"
"digest = dcb2d4acdfc14f7fee21e64bff11892c"
"Content_md5 =  3LLUrN/BT3/uIeZL/xGJLA=="
"So far so good"
"Generated url:"
"https://storage.googleapis.com/my_bucket/?GoogleAccessId=my-bucket%40my-storage.iam.gserviceaccount.com&Expires=1630665326&Signature=jboMRUyCXsMWvE5ctNRnP4l0lR521PTEP7l1bAJDqBapOPGDT7e1L
iIOWRBP5M4Q6y4K6CN98wAKMhPW7eXx%2FrHikuGsy20qQ8qsCEtyGvuS3Q2bizG32bw%2F6VVNSCARpok0hg%2Byr6cs2LC6CNUHjQ4mH5bFpxD1L6ZdXS2hsC1LNsIzuK%2BimS1RWo950aCBzKs4oPo1cw8lJoLSrg3Y9yUi3eytRr9zarpZECpqgmQynPeGKTS9aEqVlL
B8pLQ%2FwfBkhlKYC3AAXWcd0RIJjTBTHqVaL3hw%2FrVYWn%2FKmU6U3Q%2FrwRMm3HAvyGYYpn85LxpXXZYh1mfnjjXTSI6mvA%3D%3D"
"curl -X PUT \"https://storage.googleapis.com/my_bucket/?GoogleAccessId=my-bucket%40pixy-storage.iam.gserviceaccount.com&Expires=1630665326&Signature=jboMRUyCXsMWvE5ctNRnP4l0lR521PTEP7l1bAJ
DqBapOPGDT7e1LiIOWRBP5M4Q6y4K6CN98wAKMhPW7eXx%2FrHikuGsy20qQ8qsCEtyGvuS3Q2bizG32bw%2F6VVNSCARpok0hg%2Byr6cs2LC6CNUHjQ4mH5bFpxD1L6ZdXS2hsC1LNsIzuK%2BimS1RWo950aCBzKs4oPo1cw8lJoLSrg3Y9yUi3eytRr9zarpZECpqgmQy
nPeGKTS9aEqVlLB8pLQ%2FwfBkhlKYC3AAXWcd0RIJjTBTHqVaL3hw%2FrVYWn%2FKmU6U3Q%2FrwRMm3HAvyGYYpn85LxpXXZYh1mfnjjXTSI6mvA%3D%3D\" -H \"Content-Type: plain/text\"  -H Content-length:51  -H \"Content-MD5: 3LLUrN/BT
3/uIeZL/xGJLA==\"  --upload-file hello_world.txt "
<?xml version='1.0' encoding='UTF-8'?><Error><Code>MalformedBucketConfiguration</Code><Message>The XML you provided was not well-formed or did not validate against our published schema.</Message><Details>F
ailed to parse bucket creation configuration Start
Hello World
Hello World
Hello World
End
</Details></Error>

为什么这不起作用？

我在 Windows 上使用 ruby 2.7。

Answer 1

您生成的签名 URL https://storage.googleapis.com/my_bucket/?GoogleAccessId=... 不正确，因为它缺少对象的路径。这就是错误消息指示 GCS 认为您正在创建存储桶而不是上传对象的原因。

发生这种情况是因为您将 nil 作为路径传递。尝试指定路径：

sign = Google::Cloud::Storage::File::SignerV2.from_bucket bucket, "my_object"

Ruby 测试脚本使用 curl 命令将文件上传到 gcs 失败

Ruby test script upload file to gcs fails with curl command

ruby

curl

google-cloud-storage