http
(图片来自互联网)
如果一个人想学习爬虫技术,我会首先推荐他学会使用httpbin!
httpbin(官网|github)是一个很不错测试工具,你可以放心大胆的黑他,而不用担心他报复你。他有点像一个蜜罐,时刻等待着你的光临,然后根据你的请求,给你返回你想要的东西
相同推荐
类似这种能够很方便调试接口的还有很多,但是无疑还是httpbin最好用。
requestb
putsreq
ttpresponder
runscope
如何使用
使用你擅长的语言,与它的接口互动。
接口
接口列表
Endpoint | Description |
/ | This page. |
/ip | Returns Origin IP. |
/user-agent | Returns user-agent. |
/headers | Returns header dict. |
/get | Returns GET data. |
/post | Returns POST data. |
/patch | Returns PATCH data. |
/put | Returns PUT data. |
/delete | Returns DELETE data |
/gzip | Returns gzip-encoded data. |
/deflate | Returns deflate-encoded data. |
/status/:code | Returns given HTTP Status code. |
/response-headers | Returns given response headers. |
/redirect/:n | 302 Redirects n times. |
/redirect-to?url=foo | 302 Redirects to the foo URL. |
/relative-redirect/:n | 302 Relative redirects n times. |
/cookies | Returns cookie data. |
/cookies/set?name=value | Sets one or more simple cookies. |
/cookies/delete?name | Deletes one or more simple cookies. |
/basic-auth/:user/:passwd | Challenges HTTPBasic Auth. |
/hidden-basic-auth/:user/:passwd | 404'd BasicAuth. |
/digest-auth/:qop/:user/:passwd | Challenges HTTP Digest Auth. |
/stream/:n | Streams n – 100 lines. |
/delay/:n | Delays responding for n – 10 seconds. |
/drip | Drips data over a duration after an optional initial delay, then (optionally) returns with the given status code. |
/range/:n | Streams n bytes, and allows specifying a Range header to select a subset of the data. Accepts a chunk_size and request duration parameter. |
/html | Renders an HTML Page. |
/robots.txt | Returns some robots.txt rules. |
/deny | Denied by robots.txt file. |
/cache | Returns 200 unless an If-Modified-Since or If-None-Match header is provided, when it returns a 304. |
/cache/:n | Sets a Cache-Control header for n seconds. |
/bytes/:n | Generates n random bytes of binary data, accepts optional seed integer parameter. |
/stream-bytes/:n | Streams n random bytes of binary data, accepts optional seed and chunk_size integer parameters. |
/links/:n | Returns page containing n HTML links. |
/forms/post | HTML form that submits to /post |
/xml | Returns some XML |
/encoding/utf8 | Returns page containing UTF-8 data. |
## 简单例子
```python
import requests
s = requests.Session()
print s.get('http://httpbin.org/ip').text
print s.get('http://httpbin.org/get').json()
print s.post('http://httpbin.org/post', {'key':'value'},headers={'user-agent':'LAOGAO'}).text
print s.get('http://httpbin.org/status/404').status_code
print s.get('http://httpbin.org/html').text
print s.get('http://httpbin.org/deny').text
```
## 技巧
httpbin给我们提供了很多HTTP基本的操作,如果你的APP组问你要接口,你手头又很忙,怎么办?当然是用httpbin按照接口文档先给他们快速写几个丢过去。
## 部署
httpbin使用flask编写,可以参考[使用nginx + uWSGI运行flask程序][9],不过在此OZABC又学会了一种新的方法 -- 使用gunicorn运行flask。
> gunicorn又是一个 Python WSGI HTTP Server for UNIX!
```bash
# 如果已经安装PIP,请忽略此步骤
cd ~ && wget -O - "https://bootstrap.pypa.io/get-pip.py" | python
# 安装httpbin
pip install httpbin
# 安装服务器软件gunicorn
pip install gunicorn
# 运行httpbin,在8000端口
gunicorn -b :8000 httpbin:app
# [2015-06-27 03:16:52 +0000] [18568] [INFO] Starting gunicorn 19.3.0
# [2015-06-27 03:16:52 +0000] [18568] [INFO] Listening at: http://0.0.0.0:8000 (18568)
# [2015-06-27 03:16:52 +0000] [18568] [INFO] Using worker: sync
# [2015-06-27 03:16:52 +0000] [18573] [INFO] Booting worker with pid: 18573
```
接下来访问你的`IP:8000`,不出意外,你就能看到与官网一样的功能了!
![httpbin on localhost][10]
**使用文档**
[gunicorn文档][11]
[pip文档][12]
## 使用PHPSTORM RESTful调试
在 *Tools | Test RESTful* 中可以找到Test RESTful工具,可以方便的调试接口。
![RESTful][13]
[1]: https://blog.phpgao.com/usr/uploads/2015/06/727407702.jpg
[2]: http://httpbin.org
[3]: https://github.com/Runscope/httpbin
[4]: http://requestb.in
[5]: http://putsreq.com
[6]: http://httpresponder.com
[7]: https://www.runscope.com
[8]: https://github.com/Runscope/httpbin#endpoints
[9]: https://blog.phpgao.com/nginx_uwsgi_flask.html
[10]: https://blog.phpgao.com/usr/uploads/2015/06/2402402771.png
[11]: http://docs.gunicorn.org/en/latest/index.html
[12]: http://pip.readthedocs.org/en/latest/index.html
[13]: https://blog.phpgao.com/usr/uploads/2015/06/4153390953.png
如若转载,请注明出处:https://www.ozabc.com/jianzhan/36695.html