0
0

Java开源工具库使用之httpclient

文章目录

前言

HttpClient 成立于2001年，是 Apache Jakarta Commons 项目下的子项目，2004 年离开 Commons，提升成为一个单独的 Jakarta 项目。2005 年，Jakarta 创建了 HttpComponents 项目，目标是开发 HttpClient 3.x 的继任者。2007 年，Commons 项目，也就是 HttpClient 项目的发源地，离开了 Jakarta， 成为了1个新的顶级项目。不久之后，HttpComponents 也离开了 Jakarta， 成为一个独立的顶级项目，负责维护 HttpClient 的工作。

  • HttpClient 提供了高效、最新、功能丰富的支持 HTTP 协议的客户端编程工具包，支持最新版本的 HTTP 协议。
  • HttpComponents 项目，包含 HttpClient 和 HttpCore, AsyncClient 三大模块，提供了更好的性能和更大的灵活性。
  • HttpClient 是依赖于 HttpCore 的，最新的 HttpClient 版本为 5.2
  • HttpClient 是以 3.1 版本为分隔，大版本之间用法有很多不同
  • 最新文档地址：https://hc.apache.org/httpcomponents-client-5.2.x/index.html
  • 旧版文档地址：https://hc.apache.org/httpclient-legacy/userguide.html
  • github 地址：https://github.com/apache/httpcomponents-client
  • pom 依赖<!-- 最新版本5 --><dependency><groupId>org.apache.httpcomponents.client5</groupId><artifactId>httpclient5</artifactId><version>5.2.1</version></dependency><!-- 版本4 --><dependency><groupId>org.apache.httpcomponents</groupId><artifactId>httpclient</artifactId><version>4.5.13</version></dependency><!-- 旧版本3，07年后没更新 --><dependency><groupId>commons-httpclient</groupId><artifactId>commons-httpclient</artifactId><version>3.1</version></dependency>

一、简单使用

1.1 get 请求

String url ="http://httpbin.org/get";try(finalCloseableHttpClient httpclient =HttpClients.createDefault()){finalHttpGet httpget =newHttpGet(url);// Create a custom response handlerfinalHttpClientResponseHandler<String> responseHandler = response ->{finalint status = response.getCode();if(status >=HttpStatus.SC_SUCCESS&& status <HttpStatus.SC_REDIRECTION){finalHttpEntity entity = response.getEntity();try{return entity !=null?EntityUtils.toString(entity):null;}catch(finalParseException ex){thrownewClientProtocolException(ex);}}else{thrownewClientProtocolException("Unexpected response status: "+ status);}};finalString responseBody = httpclient.execute(httpget, responseHandler);System.out.println(responseBody);}

1.2 post 简单表单请求

String url ="http://httpbin.org/post";String username ="root";String loginPw ="";try(finalCloseableHttpClient httpclient =HttpClients.createDefault()){finalHttpPost httppost =newHttpPost(url);finalList<NameValuePair> params =newArrayList<>();
    params.add(newBasicNameValuePair("username", username));
    params.add(newBasicNameValuePair("password", loginPw));
    httppost.setEntity(newUrlEncodedFormEntity(params));// Create a custom response handlerfinalHttpClientResponseHandler<String> responseHandler = response ->{finalint status = response.getCode();if(status >=HttpStatus.SC_SUCCESS&& status <HttpStatus.SC_REDIRECTION){finalHttpEntity entity = response.getEntity();try{return entity !=null?EntityUtils.toString(entity):null;}catch(finalParseException ex){thrownewClientProtocolException(ex);}}else{thrownewClientProtocolException("Unexpected response status: "+ status);}};finalString responseBody = httpclient.execute(httppost, responseHandler);System.out.println(responseBody);}

1.3 表单上传文件

finalHttpPost httppost =newHttpPost(url);finalMultipartEntityBuilder builder =MultipartEntityBuilder.create();
builder.addTextBody("username", username);
builder.addTextBody("password", password);
builder.addBinaryBody("file",newFile("src/test/resources/test.txt"),ContentType.APPLICATION_OCTET_STREAM,"test.txt");finalHttpEntity multipart = builder.build();

httppost.setEntity(multipart);

1.4 上传 json 数据

finalHttpPost httppost =newHttpPost(url);

httppost.setHeader("Accept","application/json");
httppost.setHeader("Content-type","application/json");finalString json ="{\"id\":1,\"name\":\"John\"}";finalStringEntity stringEntity =newStringEntity(json);
httppost.setEntity(stringEntity);

二、高级用法

2.1 超时和重试

超时控制可以通过 RequestConfig 这个类控制

String url ="http://httpbin.org/get";RequestConfig requestConfig =RequestConfig.custom().setConnectionRequestTimeout(Timeout.ofSeconds(100L))//连接请求超时, 0为无限。默认值：3分钟。.setResponseTimeout(Timeout.ofSeconds(600L))// 响应超时时间，0为无限。带有消息复用的HTTP传输可能不支持响应超时.build();try(finalCloseableHttpClient httpclient =HttpClients.createDefault()){finalHttpGet httpGet =newHttpGet(url);
    httpGet.setConfig(requestConfig);finalHttpClientResponseHandler<String> responseHandler = response ->{finalint status = response.getCode();if(status >=HttpStatus.SC_SUCCESS&& status <HttpStatus.SC_REDIRECTION){finalHttpEntity entity = response.getEntity();try{return entity !=null?EntityUtils.toString(entity):null;}catch(finalParseException ex){thrownewClientProtocolException(ex);}}else{thrownewClientProtocolException("Unexpected response status: "+ status);}};
    httpclient.execute(httpGet, responseHandler);}

重试，默认重试策略为最大次数1次，重试间隔为1秒。

String url ="http://httpbin.org/get";try(finalCloseableHttpClient httpclient =HttpClients.custom().setRetryStrategy(newDefaultHttpRequestRetryStrategy(3,TimeValue.ofSeconds(20L))).build()){finalHttpGet httpGet =newHttpGet(url);finalHttpClientResponseHandler<String> responseHandler = response ->{finalint status = response.getCode();if(status >=HttpStatus.SC_SUCCESS&& status <HttpStatus.SC_REDIRECTION){finalHttpEntity entity = response.getEntity();try{return entity !=null?EntityUtils.toString(entity):null;}catch(finalParseException ex){thrownewClientProtocolException(ex);}}else{thrownewClientProtocolException("Unexpected response status: "+ status);}};
    httpclient.execute(httpGet, responseHandler);}

2.2 Cookie

HttpClients.createDefault 已经内置默认 Cookie 管理器可以用来携带 Cookie 访问

String url ="http://httpbin.org/cookies/set/foo/bar";String url2 ="http://httpbin.org/cookies";try(finalCloseableHttpClient httpclient =HttpClients.createDefault()){finalHttpGet httpGet =newHttpGet(url);finalHttpClientResponseHandler<String> responseHandler = response ->{finalint status = response.getCode();if(status >=HttpStatus.SC_SUCCESS&& status <HttpStatus.SC_REDIRECTION){finalHttpEntity entity = response.getEntity();try{return entity !=null?EntityUtils.toString(entity):null;}catch(finalParseException ex){thrownewClientProtocolException(ex);}}else{thrownewClientProtocolException("Unexpected response status: "+ status);}};finalHttpGet httpGet2 =newHttpGet(url2);String responseBody2 = httpclient.execute(httpGet2, responseHandler);System.out.println(responseBody2);finalString responseBody = httpclient.execute(httpGet, responseHandler);System.out.println(responseBody);

    responseBody2 = httpclient.execute(httpGet2, responseHandler);System.out.println(responseBody2);}

还可以访问通过本地上下文绑定 cookie，从而获取cookie 信息

String url ="http://httpbin.org/cookies/set/foo/bar";try(finalCloseableHttpClient httpclient =HttpClients.createDefault()){// 创建一个本地的 Cookie 存储finalCookieStore cookieStore =newBasicCookieStore();finalHttpClientContext localContext =HttpClientContext.create();// 绑定 cookieStore 到 localContext
    localContext.setCookieStore(cookieStore);finalHttpGet httpget =newHttpGet(url);finalHttpClientResponseHandler<String> responseHandler = response ->{finalint status = response.getCode();if(status >=HttpStatus.SC_SUCCESS&& status <HttpStatus.SC_REDIRECTION){finalHttpEntity entity = response.getEntity();try{finalList<Cookie> cookies = cookieStore.getCookies();for(Cookie cookie : cookies){System.out.println("Local cookie: "+ cookie);}return entity !=null?EntityUtils.toString(entity):null;}catch(finalParseException ex){thrownewClientProtocolException(ex);}}else{thrownewClientProtocolException("Unexpected response status: "+ status);}};String response = httpclient.execute(httpget, localContext, responseHandler);System.out.println(response);}

2.3 拦截器

httpclient 支持通过拦截器对请求进行一定的处理，有如下几个方法添加拦截器

  • addRequestInterceptorFirst
  • addRequestInterceptorLast
  • addResponseInterceptorFirst
  • addResponseInterceptorLast
  • addExecInterceptorFirst
  • addExecInterceptorLast
  • addExecInterceptorBefore
  • addExecInterceptorAfter

添加的拦截器可分为3种类型: request, response和 exec，对应请求，响应和执行。其中Exec执行的名字在枚举ChainElement 中，在 HttpClientBuilder 类源码中，可以发现除了 CACHING 其它都可以通过配置使用，并且枚举中的顺序也是Exec执行的顺序，其中 MAIN_TRANSPORT 执行包含 request 和 response 拦截器执行

ChainElement 定义了一组可用于构建HTTP请求处理管道的元素，每个元素都可以实现特定的功能，如添加自定义HTTP头、添加身份验证信息等。

publicenumChainElement{REDIRECT,COMPRESS,BACK_OFF,RETRY,CACHING,PROTOCOL,CONNECT,MAIN_TRANSPORT}

下面是一个对官方拦截器例子修改的代码

AtomicLong count =newAtomicLong();try(finalCloseableHttpClient httpclient =HttpClients.custom().addExecInterceptorAfter(ChainElement.PROTOCOL.name(),"custom",(request, scope, chain)->{
         request.setHeader("request-id",Long.toString(count.incrementAndGet()));return chain.proceed(request, scope);}).addExecInterceptorAfter("custom","quit3rd",((request, scope, chain)->{finalHeader idHeader = request.getFirstHeader("request-id");if(idHeader !=null&&"3".equalsIgnoreCase(idHeader.getValue())){finalClassicHttpResponse response =newBasicClassicHttpResponse(HttpStatus.SC_NOT_FOUND,"Oppsie");
             response.setEntity(newStringEntity("bad luck",ContentType.TEXT_PLAIN));return response;}else{return chain.proceed(request, scope);}})).addExecInterceptorBefore(ChainElement.CONNECT.name(),"AAA",(request, scope, chain)->{System.out.println("AAA");return chain.proceed(request, scope);}).addExecInterceptorBefore("AAA","BBB",(request, scope, chain)->{System.out.println("BBB");return chain.proceed(request, scope);}).addExecInterceptorAfter("AAA","CCC",(request, scope, chain)->{System.out.println("CCC");return chain.proceed(request, scope);}).addRequestInterceptorFirst((request, entity, context)->{System.out.println("第一个request first现在获取："+ context.getAttribute("foo"));}).addRequestInterceptorFirst((request, entity, context)->{System.out.println("第二个request first, 现在设置name");
         context.setAttribute("foo","bar");}).addRequestInterceptorLast((request, entity, context)->{System.out.println("第一个request last现在获取："+ context.getAttribute("foo"));}).build()){for(int i =0; i <5; i++){finalHttpGet httpget =newHttpGet("http://httpbin.org/get");System.out.println("Executing request "+ httpget.getMethod()+" "+ httpget.getUri());

        httpclient.execute(httpget, response ->{System.out.println("----------------------------------------");System.out.println(httpget +"->"+newStatusLine(response));EntityUtils.consume(response.getEntity());returnnull;});}}

下面动图显示的是调试过程中 execChain 执行链的顺序

内置拦截器

下面是 调试过程中的request和response 拦截器，从名字就可以看出除了Main类是自定义的拦截器，其余都是自带的，其中cookie处理也是通过拦截器实现的。

请求响应拦截器

2.4 fluent API

HttpClienet 4.5 版本以上支持fluent API, 优点是代码更简洁，同时为线程安全的。

<dependency><groupId>org.apache.httpcomponents</groupId><artifactId>fluent-hc</artifactId><version>4.5.13</version></dependency>

String urlGet ="http://httpbin.org/get";String urlPost ="http://httpbin.org/post";String response =Request.Get(urlGet).addHeader("Authorization","Bear:dw").execute().handleResponse(httpResponse ->{int code = httpResponse.getStatusLine().getStatusCode();if(code ==HttpStatus.SC_SUCCESS){returnorg.apache.http.util.EntityUtils.toString(httpResponse.getEntity());}returnnull;});System.out.println(response);String result =Request.Post(urlPost).bodyForm(Form.form().add("foo","bar").build()).execute().returnContent().asString();System.out.println(result);

三、3.1旧版本使用

3.1 Get 请求

String url ="http://httpbin.com";HttpClient client =newHttpClient();GetMethod method =newGetMethod(url);
method.getParams().setParameter(HttpMethodParams.RETRY_HANDLER,newDefaultHttpMethodRetryHandler(3,false));try{// Execute the method.int statusCode = client.executeMethod(method);if(statusCode !=HttpStatus.SC_OK){System.err.println("Method failed: "+ method.getStatusLine());}// Read the response body.byte[] responseBody = method.getResponseBody();// Deal with the response.// Use caution: ensure correct character encoding and is not binary dataSystem.out.println(newString(responseBody));}catch(HttpException e){System.err.println("Fatal protocol violation: "+ e.getMessage());
    e.printStackTrace();}catch(IOException e){System.err.println("Fatal transport error: "+ e.getMessage());
    e.printStackTrace();}finally{// Release the connection.
    method.releaseConnection();}

3.2 Post 请求

String url ="http://httpbin.org/post";HttpClient client =newHttpClient();PostMethod method =newPostMethod(url);NameValuePair[] data ={newNameValuePair("user","joe"),newNameValuePair("password","bloggs")};
method.setRequestBody(data);try{int statusCode = client.executeMethod(method);if(statusCode !=HttpStatus.SC_OK){System.err.println("Method failed: "+ method.getStatusLine());}byte[] responseBody = method.getResponseBody();System.out.println(newString(responseBody));}catch(HttpException e){System.err.println("Fatal protocol violation: "+ e.getMessage());
    e.printStackTrace();}catch(IOException e){System.err.println("Fatal transport error: "+ e.getMessage());
    e.printStackTrace();}finally{
    method.releaseConnection();}

四、异步版本使用

4.1 基本请求

finalIOReactorConfig ioReactorConfig =IOReactorConfig.custom().setSoTimeout(Timeout.ofSeconds(5)).build();finalCloseableHttpAsyncClient client =HttpAsyncClients.custom().setIOReactorConfig(ioReactorConfig).build();

client.start();finalHttpHost target =newHttpHost("httpbin.org");finalString[] requestUris =newString[]{"/","/ip","/user-agent","/headers"};for(finalString requestUri: requestUris){finalSimpleHttpRequest request =SimpleRequestBuilder.get().setHttpHost(target).setPath(requestUri).build();System.out.println("请求url:"+ requestUri);finalFuture<SimpleHttpResponse> future = client.execute(SimpleRequestProducer.create(request),SimpleResponseConsumer.create(),newFutureCallback<SimpleHttpResponse>(){@Overridepublicvoidcompleted(finalSimpleHttpResponse response){System.out.println(requestUri +" 返回状态码："+ response.getCode()+",返回内容:"+ response.getBodyText());}@Overridepublicvoidfailed(finalException ex){System.out.println(request +"->"+ ex);}@Overridepublicvoidcancelled(){System.out.println(request +" cancelled");}});
    future.get();}System.out.println("Shutting down");
client.close(CloseMode.GRACEFUL);

4.2 请求流水线执行

finalMinimalHttpAsyncClient client =HttpAsyncClients.createMinimal(H2Config.DEFAULT,Http1Config.DEFAULT,IOReactorConfig.DEFAULT,PoolingAsyncClientConnectionManagerBuilder.create().setDefaultTlsConfig(TlsConfig.custom().setVersionPolicy(HttpVersionPolicy.FORCE_HTTP_1).build()).build());

client.start();finalHttpHost target =newHttpHost("httpbin.org");finalFuture<AsyncClientEndpoint> leaseFuture = client.lease(target,null);finalAsyncClientEndpoint endpoint = leaseFuture.get(30,TimeUnit.SECONDS);try{finalString[] requestUris =newString[]{"/","/ip","/user-agent","/headers"};finalCountDownLatch latch =newCountDownLatch(requestUris.length);for(finalString requestUri: requestUris){finalSimpleHttpRequest request =SimpleRequestBuilder.get().setHttpHost(target).setPath(requestUri).build();System.out.println("Executing request "+ request);
        endpoint.execute(SimpleRequestProducer.create(request),SimpleResponseConsumer.create(),newFutureCallback<SimpleHttpResponse>(){@Overridepublicvoidcompleted(finalSimpleHttpResponse response){
                    latch.countDown();System.out.println(request +"->"+newStatusLine(response));System.out.println(response.getBody());}@Overridepublicvoidfailed(finalException ex){
                    latch.countDown();System.out.println(request +"->"+ ex);}@Overridepublicvoidcancelled(){
                    latch.countDown();System.out.println(request +" cancelled");}});}
    latch.await();}finally{
    endpoint.releaseAndReuse();}System.out.println("Shutting down");
client.close(CloseMode.GRACEFUL);

参考

  1. https://hc.apache.org/httpclient-legacy/index.html
标签： java 开源 apache
本文转载自: https://blog.csdn.net/qq_23091073/article/details/129158796
版权归原作者 aabond 所有， 如有侵权，请联系我们删除。
登录后发布评论

“Java开源工具库使用之httpclient”的评论:

还没有评论
关于作者
...
overfit同步小助手
文章同步
相关阅读

机器人技术在安全领域的重要作用

安全学习记录——网络篇（一）

图腾柱PFC：HP1010为您的电动两轮车之旅提供绿色，高效，安全的动力

单元测试数据库回滚问题

这才是单元测试，之前我们都错了

【C语言基础】：字符函数和字符串函数

ChatGPT-Next-Web SSRF漏洞+XSS漏洞复现(CVE-2023-49785)

文章导航