添加链接
link之家
链接快照平台
  • 输入网页链接,自动生成快照
  • 标签化管理网页链接
self.url = 'https://www.lfd.uci.edu/~gohlke/pythonlibs/ ' # 下载地址 self.base_url = 'https://download.lfd.uci.edu/pythonlibs/r5uhg2lo/ ' # 模拟浏览器 self.headers = { 'User-Agent ': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.96 Safari/537.36 '

3.2 获取选择器与包名

解析源码与获取包名:

# 获取选择器与包名
def getLib(self):
    raw_text = requests.get(self.url, headers=self.headers).content
    seletor = etree.HTML(raw_text)
    lib_names = seletor.xpath('//ul[@class="pylibs"]//li//strong//text()')
    return seletor, lib_names

3.3 输出所有包名

一行15个包名打印:

# 输出所有包名
    def print_AllLib(self, lib_names):
        print('\n------------------共查找到' + str(len(lib_names)) + '个包------------------\n')
        for i in range(len(lib_names)):
            if i and i % 15 == 0:
                print('\n')
            print(lib_names[i], end=' ')

3.4 搜索与下载

只摘取了核心代码,下载采用调用系统命令curl与axel,请在自己的系统上安装这两个其中之一下载!

def searchDown(self, downloadMethod='axel'):
    # 此部分代码只摘取核心部分
    if int(lib_sequence) <= max:
        detail_name = detail_libs[int(lib_sequence) - 1].strip().replace('‑', '-')
        download_url = self.base_url + detail_name
        print('\n\n输入包名正确,下载地址:' + download_url + '\n')
        print("---------------------------------------开始下载---------------------------------------\n")
        if downloadMethod == 'curl':
            cmd = 'curl -O %s' % download_url
        else:
            cmd = 'axel %s' % download_url
        while True:
            cmd_res = os.system(cmd)
            if cmd_res == 0:
                print('下载成功!\n')
                break
            else:
                print('下载失败!\n')
                is_try = input('是否重试下载(y|n): ')
                if is_try == 'y' or is_try == 'Y':
                    continue
                else:
                    break
    else:
        print("\n \033[1;31m 输入序号不符合要求,请重新输入! \033[0m \n")

3.5 调用

采用axel下载:

dw = AutoDownWhl()
dw.searchDown(downloadMethod='axel')



实际运行效果显示:
因为下载使用的windows系统版本的axel命令无法访问https,
(使用axel命令下载报错,错误原因是Unable to connect to server download.lfd.uci.edu:80。使用浏览器直接访问download.lfd.uci.edu报错,但是通过查看包的下载地址,地址开头是https,使用https://www.lfd.uci.edu/就能访问该网站,而http://www.lfd.uci.edu/却无法访问。)
所以使用curl命令,修改最后一行:dw.searchDown(downloadMethod='curl')  (前提是需要把curl命令添加到系统环境变量里)

执行python AutoDownloadWhl.py
首先会列出当前搜索到的所有whl包,接下来输入下载包的名字(英文逗号分割),然后会按照顺序,挨个查找下载包,根据python版本的不同,输入需要下载的包序号,会出现该包的下载地址。
然后curl就开始下载包了,下载的包会保存到当前目录。

或者只获取包的下载地址,使用迅雷下载

完整的下载演示如下所示:
C:\Users\sandu\Anaconda3\envs\Network\python.exe D:/scrapyseleniumtest/AutoDownloadWhl.py
---------------------------------------欢迎使用Whl自助搜索下载器---------------------------------------
------------------共查找到476个包------------------
AD3 Aggdraw Aiohttp Akima APSW Arctic Aspell-python Assimulo Astropy Atom AutoPy Babel Backports.lzma Basemap Bazaar 
Bcolz Bigfloat Bintrees Bio_formats Biopython Bitarray Blaze Blist Blosc Blz Boost.Python Bottleneck Brotli brotlipy Bsddb3 
Bsdiff4 BTrees Cairocffi Cantera Carray Cartopy Casuarius cChardet Cdecimal CellCognition CellProfiler Centrosome ceODBC Cf-units Cffi 
Cftime CGAL-bindings Cgkit Chaco Chebyfit Cheetah Chompack CLD Cmapfile cmarkgfm COBRA Coverage Crc16 Crcmod Cupy 
Curses CVXcanon CVXOPT CVXPY cx_Freeze cx_Oracle Cyassimp Cyordereddict Cyrasterize Cython Cytoolz Czifile Daal4py Datrie Debug-information-files 
Delny Dipy Discretize Dnacurve Dulwich DyND ECOS Enable Enaml Entropy ETS FabIO Fann2 Fast-histogram Fastcache 
Fastparquet Fastrlock fastText fastTSNE Faulthandler Fcsfiles Fdint Ffnet Fiona Fisher Fisx FLANN Fmkr Fpzip FreeImageDLL 
FreeSASA Friture GDAL Genshi Gensim Gevent Glumpy GMPY GPy GR Grako Greenlet gRPCio GuiQwt Gvar 
H5py HDDM Heatmap Hmmlearn HoloPy HTSeq HyperSpy Ilastik Imagecodecs ImageD11 Iminuit Imread Imreg Indexed_gzip Intbitset 
IOCBio Iris Javabridge JCC JPype Jsonlib Jupyter Kapteyn Kivy Kiwisolver Kwant La Lazy_object_proxy Lazyflow Lfdfiles 
LibLAS LIBLINEAR Libpython libSBML LIBSVM Libtfr Libxml-python Lightning Line_profiler Llist Llvmlite LLVMPy Logbook Louvain-igraph lp_solve 
Lru_dict Lsqfit Lucam Lxml Lz4 Mahotas Marisa-trie MarkupSafe Mathutils Matplotlib Mayavi Menpo Mercurial MeshPy Milk 
Minepy Mistune MKL-service MKL_fft MKL_random Mlpy mmLib MMTK Mod_wsgi ModernGL Molmass Mpi4py Mplcairo Msgpack Multidict 
MultiNEAT Multiprocess mxBase MySQL-python Mysqlclient Natgrid NDimage NetCDF4 Netifaces Netpbmfile NIPY NiTime NLopt nMOLDYN Noise 
Numba Numcodecs Numexpr NumPy Numpy-quaternion numpy-stl ObsPy ODE Oiffile Open3D OpenBabel OpenCV OpenEXR OpenImageIO OpenPIV 
openTSNE Orange Osqp Oursql Pandas Param Peewee Pendulum Persistent Pgmagick Pillow Pillow-SIMD Pip Planar Pocketsphinx 
Polygon Polylearn Polymode PostgresAdapter Protobuf Psf Psutil Psycopg PuLP Py-earth Py-fcm Py-lmdb Py2exe Py_gd PyAlembic 
PyAMF PyAMG PyArrow Pyasn PyAudio PyBluez PyBox2D Pycairo Pycares PyCifRW PyCLD2 PyCluster PyCogent PyCorrFit Pycosat 
PyCryptoSat PyCUDA PycURL Pycvodes PyDbg PyDDE PyEDA PyEMD PyEphem Pyexiv2 PyFFTW PyFITS pyFLTK PyFlux pyFM 
PyFMI Pygame Pygit2 PyGraphviz PyGreSQL PyGTK PyHDF PyHook Pyhull PyICU PyISAPIe PyJNIus Pykinsol PyKSVD Pyldap 
PyLibDeconv PyLibTiff PyLZMA Pymatgen PyMC PyMca PyMCMC PyMCubes PyMedia PyMetis PyMinuit PyMix PyMOL Pymongo Pymssql 
Pymunk Pymutt PyMVPA PyODBC Pyodeint PyOpenCL PyOpenGL Pypmc Pyproj PyQPBO PyQt4 PyQwt Pyropes Pyrsistent pyRXP 
PySFML PySide pySLALIB PySparse Pyspharm Pysqlite PyStemmer PyStruct Pyswisseph PyTables Python-cjson Python-geohash Python-igraph Python-ldap Python-Levenshtein 
Python-lzf Python-LZO Python-snappy Python-sundials PythonMagick PythonNET Pytiff PyTorch PyTST PyUSB-FTDI PyViennaCL PyVRML97 PyWavelets PyWCS PyWin32 
PyWinHook Pywinpty PyX PyXML PyYAML PyZMQ Pyzopfli Qdafile Qimage2ndarray Qt_graph_helpers QuantLib QuickFIX QuTiP Rapidjson Rasterio 
Re2 Recordclass Regex ReportLab Rpy2 Rtmidi-python Rtree Ruamel.yaml Salientdetect Sasl Scandir ScientificPython Scikit-cycling Scikit-fmm Scikit-image 
Scikit-learn Scikit-misc Scikit-umfpack Scikits.ann Scikits.audiolab Scikits.delaunay Scikits.hydroclimpy Scikits.odes Scikits.samplerate Scikits.scattpy Scikits.timeseries Scikits.vectorplot Scimath SciPy Scipy-cluster 
SCS Sdtfile SendKeys Seqlearn Setproctitle SfePy Shapely SilverCity Sima Simpleaudio SimpleITK Simplejson SimpleParse Slycot Sounddevice 
spaCy Sparsesvd Spectrum Spglib SQLAlchemy Statsmodels STEPS Stratify Swiglpk Sympy TA-Lib TatSu TensorFlow Thrift ThriftPy 
Tifffile Tinyarray Tomopy Tornado Traits Transformations Triangle Trollius TWAINmodule Twisted typed_ast Uciwebauth UDUNITS Ujson Umysql 
Veusz VideoCapture Vidsrc VIGRA ViTables VPython VTK Winrandom Wordcloud Wrapt Wrf_python wxPython x86cpu Xgboost Xxhash 
XYlib-py Yappi Yarl Yt Zfec Zipline Zodbpickle Zope.interface Zs Zstd Misc 
请输入下载包的名字(英文逗号分割): SQLAlchemy
---------查找SQLAlchemy成功,正在输出包的详细信息---------
(1)SQLAlchemy‑1.2.18‑cp27‑cp27m‑win32.whl
(2)SQLAlchemy‑1.2.18‑cp27‑cp27m‑win_amd64.whl
(3)SQLAlchemy‑1.2.18‑cp35‑cp35m‑win32.whl
(4)SQLAlchemy‑1.2.18‑cp35‑cp35m‑win_amd64.whl
(5)SQLAlchemy‑1.2.18‑cp36‑cp36m‑win32.whl
(6)SQLAlchemy‑1.2.18‑cp36‑cp36m‑win_amd64.whl
(7)SQLAlchemy‑1.2.18‑cp37‑cp37m‑win32.whl
(8)SQLAlchemy‑1.2.18‑cp37‑cp37m‑win_amd64.whl
请输入下载的包名序号: 8
输入包名正确,下载地址:https://download.lfd.uci.edu/pythonlibs/r5uhg2lo/SQLAlchemy-1.2.18-cp37-cp37m-win_amd64.whl
---------------------------------------开始下载---------------------------------------
curl -O https://download.lfd.uci.edu/pythonlibs/r5uhg2lo/SQLAlchemy-1.2.18-cp37-cp37m-win_amd64.whl
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1087k  100 1087k    0     0  39631      0  0:00:28  0:00:28 --:--:-- 45299
下载成功!
是否继续下载其他库(y|n): n
---------------------------------------感谢使用---------------------------------------

修改后使用curl命令下载的代码地址:

https://files.cnblogs.com/files/sanduzxcvbnm/AutoDownloadWhl-master.7z