本文介绍如何安装 Python 3.5 并配置 Scrapy 爬虫环境，包括依赖库的安装、Python 版本切换及 Scrapy 命令使用等关键步骤。
- python -V
- yum -y update
- yum groupinstall -y development
- yum install -y zlib-dev openssl-devel sqlite-devel bzip2-devel
- wget https://www.python.org/ftp/python/3.5.2/Python-3.5.2.tar.xz
- tar -xvf Python-3.5.2.tar.xz
- cd Python-3.5.2
- ./configure
- make && make install
- mv /usr/bin/python /usr/bin/python2.6.6
- ln -s /usr/local/bin/python3.5 /usr/bin/python
- vi /usr/bin/yum(#!/usr/bin/python 改为 #!/usr/bin/python2.6.6)
- wget --no-check-certificate https://pypi.python.org/packages/source/s/setuptools/setuptools-1.4.2.tar.gz
- tar -xvf setuptools-1.4.2.tar.gz
- cd setuptools-1.4.2
- python setup.py install
- ln -s /usr/local/bin/easy_install-3.5 /usr/bin/easy_install
- easy_install pip
- curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py | python3.5
- yum -y install libxslt-devel
- yum -y install libffi-devel
- pip install scrapy
- scrapy -h
Scrapy 1.1.2 - project: tutorial
Usage:
scrapy <command> [options] [args]
Available commands:
bench Run quick benchmark test
check Check spider contracts
commands
crawl Run a spider
edit Edit spider
fetch Fetch a URL using the Scrapy downloader
genspider Generate new spider using pre-defined templates
list List available spiders
parse Parse URL (using its spider) and print the results
runspider Run a self-contained spider (without creating a project)
settings Get settings values
shell Interactive scraping console
startproject Create new project
version Print Scrapy version
view Open URL in browser, as seen by Scrapy
Use "scrapy <command> -h" to see more info about a command
Centos6.5安装Scrapy
最新推荐文章于 2023-01-06 21:19:35 发布
扫一扫
1570
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
