HTTP Post 到 API return 403 禁止访问
HTTP Post to API return 403 FORBIDDEN
我必须将 XML 文件上传到 API。这是 API 由我从 API.
的发行者处获得的签名证书保护的
现在，我有两个用例。首先，我必须从 API 下载一些文件。这与以下代码完美配合：
final Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(ip, port));
final URL url = new URL(linkToFile);
final HttpsURLConnection conn = (HttpsURLConnection) url.openConnection(proxy);
conn.setSSLSocketFactory(this.http.createSSLContext().getSocketFactory());
try (
InputStream inputStream = zipUrlConn.getInputStream();
ZipInputStream stream = new ZipInputStream(inputStream);) {
// Do stuff with ZipInputStream here
}
createSSLContext() 方法如下所示：
public SSLContext createSSLContext() throws NoSuchAlgorithmException, CertificateException, IOException, KeyStoreException,
UnrecoverableKeyException, KeyManagementException {
final KeyStore clientStore = KeyStore.getInstance("PKCS12");
clientStore.load(new FileInputStream(this.certificateResource.getFile()), this.p12PW.toCharArray());
final KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
keyManagerFactory.init(clientStore, this.p12PW.toCharArray());
final KeyManager[] keyManagers = keyManagerFactory.getKeyManagers();
final KeyStore trustStore = KeyStore.getInstance("JKS");
trustStore.load(new FileInputStream(this.trustStoreResource.getFile()), this.trustStorePW.toCharArray());
final TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
trustManagerFactory.init(trustStore);
final TrustManager[] trustManagers = trustManagerFactory.getTrustManagers();
final SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(keyManagers, trustManagers, new SecureRandom());
return sslContext;
}
我遵循了从发行者那里得到的指南，其中展示了如何使用 cUrl 命令执行此操作：
curl --cert Certificate-<id>.pem[:pem_password] https://api.url.com/
所以我基本上是在尝试在 java 中重建这个命令，这是有效的。
现在是无法正常工作的部分，即文件上传。再一次，我得到了一个我必须重建的 cUrl 命令：
curl --cert Certificate-<id>.pem[:pem_password] -F upload=@<Path_to_file>\DS<PartnerId>_<Timestamp>.XML https://api.url.com/in/upload.php
我尝试了几种方法来实现：
- "Normal" Java
首先，我用标准的HttpsURLConnection试了一下，如下：
final Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("192.168.1.25", 3128));
final HttpsURLConnection connection = (HttpsURLConnection) new URL(ARGE_UPLOAD_URL).openConnection(proxy);
connection.setSSLSocketFactory(this.http.createSSLContext().getSocketFactory());
connection.setDoOutput(true);
connection.setRequestProperty("Accept-Charset", "UTF-8");
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded;charset=UTF-8");
try (OutputStream outputStream = connection.getOutputStream()) {
outputStream.write(Files.readAllBytes(new File("src/main/resources/XML/example.xml").toPath()));
}
final InputStream result = connection.getInputStream();
但这总是导致 java.io.IOException: Server returned HTTP response code: 403 for URL: https://api.url.com/in/upload.php，即使我使用相同的配置，我可以从 API.
下载
- Apache HttpClient
我发现一些资源声称 HttpClient 更容易配置和使用，所以我尝试了一下：
final CloseableHttpClient httpClient = HttpClientBuilder
.create()
.setSSLContext(this.http.createSSLContext())
.setProxy(new HttpHost(InetAddress.getByName(ip), port))
.build();
final HttpEntity requestEntity = MultipartEntityBuilder
.create()
.addBinaryBody("upload=@example.xml", new File("src/main/resources/XML/example.xml")) // Hardcoded for testing
.build();
final HttpPost post = new HttpPost(https://api.url.com/in/upload.php);
post.setEntity(requestEntity);
try (CloseableHttpResponse response = httpClient.execute(post)) {
this.logger.info(response.getStatusLine().toString());
EntityUtils.consume(response.getEntity());
}
导致 HTTP/1.1 403 FORBIDDEN
- HttpClient（FileEntity 而不是 MultipartEntity）
作为最后一件事，我尝试了 FileEntity:
final HttpPost httpPost = new HttpPost(https://api.url.com/in/upload.php);
httpPost.addHeader("content-type", "application/x-www-form-urlencoded;charset=utf-8");
final FileEntity fileEntity = new FileEntity(new File("src/main/resources/XML/example.xml"));
httpPost.setEntity(fileEntity);
System.out.println("executing request " + httpPost.getRequestLine() + httpPost.getConfig());
try (CloseableHttpResponse response = httpClient.execute(httpPost)) {
final HttpEntity responseEntity = response.getEntity();
System.out.println("Status: " + response.getStatusLine());
if (responseEntity != null) {
System.out.println("Entity: " + EntityUtils.toString(responseEntity));
}
}
导致 Status: HTTP/1.1 403 FORBIDDEN
我只是不明白，尽管使用完全相同的配置，我怎么能从 API 下载但不能上传到它。
如果您需要更多信息，我很乐意提供。
编辑
根据oli的建议，我使用Fiddler 来捕获HTTPS 请求。这是方法 1 的结果（正常 Java）：
POST https://hrbaxml.arbeitsagentur.de/in/upload.php HTTP/1.1
Accept-Charset: utf-8
Accept: */*
Content-Type: application/x-www-form-urlencoded;charset=utf-8
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36
Host: hrbaxml.arbeitsagentur.de
Connection: keep-alive
Content-Length: 6738
这是通过Google手动上传的结果 Chrome:
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7
Cache-Control: max-age=0
Connection: keep-alive
Content-Length: 6948
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryxZxIJ5h19MEFbZQs
Cookie: cookie_consent=accepted
Host: hrbaxml.arbeitsagentur.de
Origin: https://hrbaxml.arbeitsagentur.de
Referer: https://hrbaxml.arbeitsagentur.de/in/
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36
编辑 2
补充一下，这是使用方法 2（带有 MultipartEntity 的 HttpClient）的结果：
POST https://hrbaxml.arbeitsagentur.de/in/upload.php HTTP/1.1
Content-Length: 7025
Content-Type: multipart/form-data; boundary=1sZvrqcGe-FuQ3r-_fFgt2SJtZ5_yo7Pfvq_
Host: hrbaxml.arbeitsagentur.de
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.5 (Java/1.8.0_161)
Accept-Encoding: gzip,deflate
--1sZvrqcGe-FuQ3r-_fFgt2SJtZ5_yo7Pfvq_
Content-Disposition: form-data; name="DSV000306700_2018-08-23_09-00-00.xml"; filename="DSV000306700_2018-08-23_09-00-00.xml"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary
编辑 3
我尝试从 Chrome 请求中复制所有 HTTP Headers，这样我来自 Java 的请求看起来像这样：
POST https://hrbaxml.arbeitsagentur.de/in/upload.php HTTP/1.1
Accept: Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7
Cache-Control: max-age=0
Content-Type: multipart/form-data; boundary=1535095530678
Cookie: cookie_consent=accepted
Referer: https://hrbaxml.arbeitsagentur.de/in/
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36
Host: hrbaxml.arbeitsagentur.de
Connection: keep-alive
Content-Length: 6944
--1535095530678
Content-Disposition: form-data; name="uploadFile"; filename="DSV000306700_2018-08-23_09-00-00.xml"
Content-Type: application/xml
Content-Transfer-Encoding: binary
.. xml data ..
--1535095530678--
但仍然没有成功。还有其他可能的解决方案吗？也许不是上传的问题，而是其他问题？
这个信息来自服务器端，所以你应该向服务提供商寻求帮助，也许上传需要授权
我会捕获您使用 Java-Application 发送到服务器的 HTTP 请求（f.e。使用 Wireshark），并将其与您从浏览器发送的 HTTP 请求进行比较（您可以使用内置的浏览器工具轻松捕获它，请尝试按 F12)。
我 100% 确定您会看到一些差异，这对我来说总是有效的。
编辑：
还有一个可能的问题。请尝试添加
connection.addRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:25.0) Gecko/20100101 Firefox/25.0");
或您的浏览器在您的第一个实现中发送的同一个。还要确保你的 SSL 证书和加密算法没有问题（你使用默认的算法，你的情况就是它）。此外（如果没有帮助）您还可以检查协商握手密钥的密钥长度。
好吧，我终于想通了。正如预期的那样，错误出现在我的 POST 请求中。首先，这是工作代码（使用方法 2）：
final CloseableHttpClient httpClient = HttpClientBuilder
.create()
.setSSLContext(this.http.createSSLContext())
.setProxy(new HttpHost(InetAddress.getByName(ip), port))
.build();
final HttpEntity requestEntity = MultipartEntityBuilder
.create()
.addBinaryBody("upload", new File("src/main/resources/XML/example.xml")) // Hardcoded for testing
.build();
final HttpPost post = new HttpPost(https://api.url.com/in/upload.php);
post.setEntity(requestEntity);
try (CloseableHttpResponse response = httpClient.execute(post)) {
this.logger.info(response.getStatusLine().toString());
EntityUtils.consume(response.getEntity());
}
如您所见，唯一发生变化的行是 .addBinaryBody("upload", new File("src/main/resources/XML/example.xml"))。 "upload" 至关重要，因为我试图重建的 cUrl 调用具有 -F 标志。来自 cUrl manual:
(HTTP SMTP IMAP) For HTTP protocol family, this lets curl emulate a filled-in form in which a user has pressed the submit button. This causes curl to POST data using the Content-Type multipart/form-data according to RFC 2388.
然后每个部分都包含文件的名称和数据。我正在使用的 API 依赖于名称 "upload" 来处理请求。否则它不知道该做什么 returns 403 FORBIDDEN.
我必须将 XML 文件上传到 API。这是 API 由我从 API.
的发行者处获得的签名证书保护的现在，我有两个用例。首先，我必须从 API 下载一些文件。这与以下代码完美配合：
final Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(ip, port));
final URL url = new URL(linkToFile);
final HttpsURLConnection conn = (HttpsURLConnection) url.openConnection(proxy);
conn.setSSLSocketFactory(this.http.createSSLContext().getSocketFactory());
try (
InputStream inputStream = zipUrlConn.getInputStream();
ZipInputStream stream = new ZipInputStream(inputStream);) {
// Do stuff with ZipInputStream here
}
createSSLContext() 方法如下所示：
public SSLContext createSSLContext() throws NoSuchAlgorithmException, CertificateException, IOException, KeyStoreException,
UnrecoverableKeyException, KeyManagementException {
final KeyStore clientStore = KeyStore.getInstance("PKCS12");
clientStore.load(new FileInputStream(this.certificateResource.getFile()), this.p12PW.toCharArray());
final KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
keyManagerFactory.init(clientStore, this.p12PW.toCharArray());
final KeyManager[] keyManagers = keyManagerFactory.getKeyManagers();
final KeyStore trustStore = KeyStore.getInstance("JKS");
trustStore.load(new FileInputStream(this.trustStoreResource.getFile()), this.trustStorePW.toCharArray());
final TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
trustManagerFactory.init(trustStore);
final TrustManager[] trustManagers = trustManagerFactory.getTrustManagers();
final SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(keyManagers, trustManagers, new SecureRandom());
return sslContext;
}
我遵循了从发行者那里得到的指南，其中展示了如何使用 cUrl 命令执行此操作：
curl --cert Certificate-<id>.pem[:pem_password] https://api.url.com/
所以我基本上是在尝试在 java 中重建这个命令，这是有效的。
现在是无法正常工作的部分，即文件上传。再一次，我得到了一个我必须重建的 cUrl 命令：
curl --cert Certificate-<id>.pem[:pem_password] -F upload=@<Path_to_file>\DS<PartnerId>_<Timestamp>.XML https://api.url.com/in/upload.php
我尝试了几种方法来实现：
- "Normal" Java
首先，我用标准的HttpsURLConnection试了一下，如下：
final Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("192.168.1.25", 3128));
final HttpsURLConnection connection = (HttpsURLConnection) new URL(ARGE_UPLOAD_URL).openConnection(proxy);
connection.setSSLSocketFactory(this.http.createSSLContext().getSocketFactory());
connection.setDoOutput(true);
connection.setRequestProperty("Accept-Charset", "UTF-8");
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded;charset=UTF-8");
try (OutputStream outputStream = connection.getOutputStream()) {
outputStream.write(Files.readAllBytes(new File("src/main/resources/XML/example.xml").toPath()));
}
final InputStream result = connection.getInputStream();
但这总是导致 java.io.IOException: Server returned HTTP response code: 403 for URL: https://api.url.com/in/upload.php，即使我使用相同的配置，我可以从 API.
- Apache HttpClient
我发现一些资源声称 HttpClient 更容易配置和使用，所以我尝试了一下：
final CloseableHttpClient httpClient = HttpClientBuilder
.create()
.setSSLContext(this.http.createSSLContext())
.setProxy(new HttpHost(InetAddress.getByName(ip), port))
.build();
final HttpEntity requestEntity = MultipartEntityBuilder
.create()
.addBinaryBody("upload=@example.xml", new File("src/main/resources/XML/example.xml")) // Hardcoded for testing
.build();
final HttpPost post = new HttpPost(https://api.url.com/in/upload.php);
post.setEntity(requestEntity);
try (CloseableHttpResponse response = httpClient.execute(post)) {
this.logger.info(response.getStatusLine().toString());
EntityUtils.consume(response.getEntity());
}
导致 HTTP/1.1 403 FORBIDDEN
- HttpClient（FileEntity 而不是 MultipartEntity）
作为最后一件事，我尝试了 FileEntity:
final HttpPost httpPost = new HttpPost(https://api.url.com/in/upload.php);
httpPost.addHeader("content-type", "application/x-www-form-urlencoded;charset=utf-8");
final FileEntity fileEntity = new FileEntity(new File("src/main/resources/XML/example.xml"));
httpPost.setEntity(fileEntity);
System.out.println("executing request " + httpPost.getRequestLine() + httpPost.getConfig());
try (CloseableHttpResponse response = httpClient.execute(httpPost)) {
final HttpEntity responseEntity = response.getEntity();
System.out.println("Status: " + response.getStatusLine());
if (responseEntity != null) {
System.out.println("Entity: " + EntityUtils.toString(responseEntity));
}
}
导致 Status: HTTP/1.1 403 FORBIDDEN
我只是不明白，尽管使用完全相同的配置，我怎么能从 API 下载但不能上传到它。
如果您需要更多信息，我很乐意提供。
编辑
根据oli的建议，我使用Fiddler 来捕获HTTPS 请求。这是方法 1 的结果（正常 Java）：
POST https://hrbaxml.arbeitsagentur.de/in/upload.php HTTP/1.1
Accept-Charset: utf-8
Accept: */*
Content-Type: application/x-www-form-urlencoded;charset=utf-8
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36
Host: hrbaxml.arbeitsagentur.de
Connection: keep-alive
Content-Length: 6738
这是通过Google手动上传的结果 Chrome:
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7
Cache-Control: max-age=0
Connection: keep-alive
Content-Length: 6948
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryxZxIJ5h19MEFbZQs
Cookie: cookie_consent=accepted
Host: hrbaxml.arbeitsagentur.de
Origin: https://hrbaxml.arbeitsagentur.de
Referer: https://hrbaxml.arbeitsagentur.de/in/
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36
编辑 2
补充一下，这是使用方法 2（带有 MultipartEntity 的 HttpClient）的结果：
POST https://hrbaxml.arbeitsagentur.de/in/upload.php HTTP/1.1
Content-Length: 7025
Content-Type: multipart/form-data; boundary=1sZvrqcGe-FuQ3r-_fFgt2SJtZ5_yo7Pfvq_
Host: hrbaxml.arbeitsagentur.de
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.5 (Java/1.8.0_161)
Accept-Encoding: gzip,deflate
--1sZvrqcGe-FuQ3r-_fFgt2SJtZ5_yo7Pfvq_
Content-Disposition: form-data; name="DSV000306700_2018-08-23_09-00-00.xml"; filename="DSV000306700_2018-08-23_09-00-00.xml"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary
编辑 3
我尝试从 Chrome 请求中复制所有 HTTP Headers，这样我来自 Java 的请求看起来像这样：
POST https://hrbaxml.arbeitsagentur.de/in/upload.php HTTP/1.1
Accept: Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7
Cache-Control: max-age=0
Content-Type: multipart/form-data; boundary=1535095530678
Cookie: cookie_consent=accepted
Referer: https://hrbaxml.arbeitsagentur.de/in/
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36
Host: hrbaxml.arbeitsagentur.de
Connection: keep-alive
Content-Length: 6944
--1535095530678
Content-Disposition: form-data; name="uploadFile"; filename="DSV000306700_2018-08-23_09-00-00.xml"
Content-Type: application/xml
Content-Transfer-Encoding: binary
.. xml data ..
--1535095530678--
但仍然没有成功。还有其他可能的解决方案吗？也许不是上传的问题，而是其他问题？
这个信息来自服务器端，所以你应该向服务提供商寻求帮助，也许上传需要授权
我会捕获您使用 Java-Application 发送到服务器的 HTTP 请求（f.e。使用 Wireshark），并将其与您从浏览器发送的 HTTP 请求进行比较（您可以使用内置的浏览器工具轻松捕获它，请尝试按 F12)。
我 100% 确定您会看到一些差异，这对我来说总是有效的。
编辑：
还有一个可能的问题。请尝试添加
connection.addRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:25.0) Gecko/20100101 Firefox/25.0");
或您的浏览器在您的第一个实现中发送的同一个。还要确保你的 SSL 证书和加密算法没有问题（你使用默认的算法，你的情况就是它）。此外（如果没有帮助）您还可以检查协商握手密钥的密钥长度。
好吧，我终于想通了。正如预期的那样，错误出现在我的 POST 请求中。首先，这是工作代码（使用方法 2）：
final CloseableHttpClient httpClient = HttpClientBuilder
.create()
.setSSLContext(this.http.createSSLContext())
.setProxy(new HttpHost(InetAddress.getByName(ip), port))
.build();
final HttpEntity requestEntity = MultipartEntityBuilder
.create()
.addBinaryBody("upload", new File("src/main/resources/XML/example.xml")) // Hardcoded for testing
.build();
final HttpPost post = new HttpPost(https://api.url.com/in/upload.php);
post.setEntity(requestEntity);
try (CloseableHttpResponse response = httpClient.execute(post)) {
this.logger.info(response.getStatusLine().toString());
EntityUtils.consume(response.getEntity());
}
如您所见，唯一发生变化的行是 .addBinaryBody("upload", new File("src/main/resources/XML/example.xml"))。 "upload" 至关重要，因为我试图重建的 cUrl 调用具有 -F 标志。来自 cUrl manual:
(HTTP SMTP IMAP) For HTTP protocol family, this lets curl emulate a filled-in form in which a user has pressed the submit button. This causes curl to POST data using the Content-Type multipart/form-data according to RFC 2388.
然后每个部分都包含文件的名称和数据。我正在使用的 API 依赖于名称 "upload" 来处理请求。否则它不知道该做什么 returns 403 FORBIDDEN.