|jAw@sdZddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl m Z ddlmZddlmZddlmZddlmZdd lmZmZmZmZmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#dd l$m%Z%dd l&m'Z'dd l(m)Z)m*Z*m+Z+m,Z,dlZ-ej.Z/ddddiZ0ej1dkrddZ2ddZddZ3ddZ4dddZ5dd Z6d!d"Z7d#d$Z8d%d&Z9d'd(Z:d)d*Z;dd+d,Z<d-d.Z=d/d0Z>d1d2Z?d3d4Z@d5d6ZAd7d8ZBd9d:ZCd;d<ZDeEd=d>ZFd?d@ZGdAdBZHdCdDZIdEdFZJdGdHZKdIdJZLejMdKdLZNdMdNZOddOdPZPdQdRZQdSdTdUZRdVdWZSdXdYZTdZjUd[ZVeVd\ZWeVd]ZXd^d_ZYd`daZZdbdcZ[ej\ddZ]ej\deZ^dfdgZ_dhdiZ`djdkZadS)mz requests.utils ~~~~~~~~~~~~~~ This module provides utility functions that are used within Requests that are also useful for external consumption. N) OrderedDict) __version__)certs)to_native_string)parse_http_list)quoteurlparsebytesstrunquote getproxies proxy_bypass urlunparse basestring integer_typesis_py3proxy_bypass_environmentgetproxies_environmentMapping)cookiejar_from_dict)CaseInsensitiveDict) InvalidURL InvalidHeaderFileModeWarningUnrewindableBodyError.netrc_netrchttpPhttpsiwin32cCsFy%trddl}n ddl}Wntk r=dSYnXyK|j|jd}t|j|dd}|j|dd}Wntk rdSYnX| s| rdS|j d}x||D]t}|dkrd|krd S|j dd }|j d d }|j d d}t j ||t j rd SqWdS)NrFz;Software\Microsoft\Windows\CurrentVersion\Internet SettingsZ ProxyEnableZ ProxyOverride;z.Tz\.*z.*?)rwinreg_winreg ImportErrorOpenKeyHKEY_CURRENT_USERint QueryValueExOSErrorsplitreplacerematchI)hostr&ZinternetSettingsZ proxyEnableZ proxyOverridetestr52/tmp/pip-build-1hwfuv5s/requests/requests/utils.pyproxy_bypass_registry0s8           r7cCs!trt|St|SdS)zReturn True, if the host should be bypassed. Checks proxy settings gathered from the environment, if specified, or the registry. N)rrr7)r3r5r5r6rWs  rcCst|dr|j}|S)z/Returns an internal sequence dictionary update.items)hasattrr8)dr5r5r6dict_to_sequencecs r;cCsd}d}t|dr*t|}nt|drE|j}nht|dry|j}Wntjk r{Yn2Xtj|j}d|jkrt j dt t|drhy|j }Wn*t tfk r|dk r|}YnpXt|drh|dkrhy3|jdd |j }|j|pBdWnt tfk rgd}YnX|dkrzd}td||S) Nr__len__lenfilenoba%Requests has determined the content-length for this request using the binary size of the file: however, the file has been opened in text mode (i.e. without the 'b' flag in the mode). This may lead to an incorrect content-length. In Requests 3.0, support will be removed for files in text mode.tellseek)r9r=r>ioUnsupportedOperationosfstatst_sizemodewarningswarnrr@r-IOErrorrAmax)o total_lengthcurrent_positionr>r5r5r6 super_lenls@      rPFcCstjjd}|dk r*|f}nddtD}y:ddlm}m}d}xV|D]N}ytjj|}Wntk rdSYnXtjj |rc|}PqcW|dkrdSt |} d} t |t r| j d} | jj| d} yG||j| } | rQ| dr9dnd } | | | d fSWn!|tfk ru|rqYnXWnttfk rYnXdS) z;Returns the Requests tuple auth for a given url from netrc.ZNETRCNcss|]}dj|VqdS)z~/{}N)format).0fr5r5r6 sz!get_netrc_auth..r)netrcNetrcParseError:asciirrB)rEenvironget NETRC_FILESrUrVpath expanduserKeyErrorexistsr isinstancer decodenetlocr.authenticatorsrKr(AttributeError)url raise_errorsZ netrc_fileZnetrc_locationsrUrV netrc_pathrSlocrisplitstrr3rlogin_ir5r5r6get_netrc_auths@        rlcCs[t|dd}|rWt|trW|ddkrW|ddkrWtjj|SdS)z0Tries to guess the filename of the given object.nameNr)getattrr`rrEr\basename)objrmr5r5r6guess_filenames%rtcCstjj|r|Stjj|\}}xJ|rztjj| rztjj|\}}dj||g}q1Wtj|s|Stj|}||jkr|St j }tjj||jd}tjj|s|j |d|}|S)zReplace nonexistent paths that look like they refer to a member of a zip archive with the location of an extracted copy of the target, or else just return the provided path unchanged. /r\) rEr\r_r.joinzipfile is_zipfileZipFilenamelisttempfile gettempdirextract)r\archivememberprefixZzip_filetmpZextracted_pathr5r5r6extract_zipped_pathss  rcCsA|dkrdSt|ttttfr7tdt|S)aTake an object and test to see if it can be represented as a dictionary. Unless it can not be represented as such, return an OrderedDict, e.g., :: >>> from_key_val_list([('key', 'val')]) OrderedDict([('key', 'val')]) >>> from_key_val_list('string') Traceback (most recent call last): ... ValueError: cannot encode objects that are not 2-tuples >>> from_key_val_list({'key': 'val'}) OrderedDict([('key', 'val')]) :rtype: OrderedDict Nz+cannot encode objects that are not 2-tuples)r`r r boolr+ ValueErrorr)valuer5r5r6from_key_val_list s   rcCs\|dkrdSt|ttttfr7tdt|trR|j}t|S)aTake an object and test to see if it can be represented as a dictionary. If it can be, return a list of tuples, e.g., :: >>> to_key_val_list([('key', 'val')]) [('key', 'val')] >>> to_key_val_list({'key': 'val'}) [('key', 'val')] >>> to_key_val_list('string') Traceback (most recent call last): ... ValueError: cannot encode objects that are not 2-tuples :rtype: list Nz+cannot encode objects that are not 2-tuples) r`r r rr+rrr8list)rr5r5r6to_key_val_list%s   rcCstg}xgt|D]Y}|dd|ddkoDdknr_t|dd}|j|qW|S)aParse lists as described by RFC 2068 Section 2. In particular, parse comma-separated lists where the elements of the list may include quoted-strings. A quoted-string could contain a comma. A non-quoted string could have quotes in the middle. Quotes are removed automatically after parsing. It basically works like :func:`parse_set_header` just that items may appear multiple times and case sensitivity is preserved. The return value is a standard :class:`list`: >>> parse_list_header('token, "quoted value"') ['token', 'quoted value'] To create a header from the :class:`list` again, use the :func:`dump_header` function. :param value: a string with a list header. :return: :class:`list` :rtype: list Nr"rprp)_parse_list_headerunquote_header_valueappend)rresultitemr5r5r6parse_list_headerCs 0rcCsi}xt|D]}d|kr2d||>> d = parse_dict_header('foo="is a fish", bar="as well"') >>> type(d) is dict True >>> sorted(d.items()) [('bar', 'as well'), ('foo', 'is a fish')] If there is no value for a key it will be `None`: >>> parse_dict_header('key_without_value') {'key_without_value': None} To create a header from the :class:`dict` again, use the :func:`dump_header` function. :param value: a string with a dict header. :return: :class:`dict` :rtype: dict =Nrrrprp)rr.r)rrrrmr5r5r6parse_dict_headercs  0rcCsw|rs|d|d ko%dknrs|dd }| sW|dddkrs|jddjddS|S) zUnquotes a header value. (Reversal of :func:`quote_header_value`). This does not use the real unquoting but what browsers are actually using for quoting. :param value: the header value to unquote. :rtype: str rrrNrBz\\\z\"rprp)r/)r is_filenamer5r5r6rs *rcCs+i}x|D]}|j||j]flagsz+]z$^<\?xml.*?encoding=["\']*(.+?)["\'>])rIrJDeprecationWarningr0compiler2findall)content charset_re pragma_rexml_rer5r5r6get_encodings_from_contentsrc Cs|jd}|dj|dd}}i}d}x|D]}|j}|rC|d}}|jd} | dkr|d| j|}|| ddj|}|||jdA}tjtjd|S)zConverts mask from /xx format to xxx.xxx.xxx.xxx Example: if mask is 24 function returns 255.255.255.0 :rtype: str lr z>I)r inet_ntoarpack)maskrr5r5r6rsrc Cs1ytj|Wntjk r,dSYnXdS)z :rtype: bool FT)rrerror) string_ipr5r5r6is_ipv4_addresss  rcCs|jddkryt|jdd}Wntk rJdSYnX|dksc|dkrgdSytj|jddWqtjk rdSYqXndSdS)zV Very simple check of the cidr format in no_proxy variable. :rtype: bool rurFrrT)countr+r.rrrr)string_networkrr5r5r6 is_valid_cidrs   rc csn|dk }|r1tjj|}|tj|}t|rt|j|rdSq|j|krdSqWn^|j}|jr|d j|j7}x3|D]+}|jj |s(|j |rdSqWt d|<yt |j}Wn!t t jfk rvd }YnXWdQRX|rdSd S) zL Returns whether we should bypass proxies or not. :rtype: bool cSs(tjj|p'tjj|jS)N)rErYrZupper)kr5r5r6sz'should_bypass_proxies..Nno_proxyTcss|]}|r|VqdS)Nr5)rRr3r5r5r6rTsz(should_bypass_proxies.. r,z:{}F)r hostnamer/r.rrrportrQendswithrrrrgaierror) rer get_proxyZ no_proxy_argparsedproxy_ipZhost_with_portr3bypassr5r5r6should_bypass_proxiess<    (      !rcCs!t|d|riStSdS)zA Return a dict of environment proxies. :rtype: dict rN)rr )rerr5r5r6get_environ_proxiessrcCs|p i}t|}|jdkrC|j|j|jdS|jd|j|jd|jdg}d}x%|D]}||krz||}PqzW|S)zSelect a proxy for the url, if applicable. :param url: The url being for the request :param proxies: A dictionary of schemes or schemes and hosts to proxy URLs Nallz://zall://)r rrZscheme)reproxiesurlparts proxy_keysproxy proxy_keyr5r5r6 select_proxys       rzpython-requestscCsd|tfS)zO Return a string representing the default user agent. :rtype: str z%s/%s)r)rmr5r5r6default_user_agent(src Cs.tdtddjd dddd iS) z9 :rtype: requests.structures.CaseInsensitiveDict z User-AgentzAccept-Encodingz, gzipdeflateAcceptz*/* Connectionz keep-alive)rr)rrrvr5r5r5r6default_headers1s  r c Cs g}d}|j|}|s%|Sxtjd|D]}y|jdd\}}Wntk r{|d}}YnXd|jdi}xb|jdD]Q}y|jd\}}Wntk rPYnX|j|||j|; rel=front; type="image/jpeg",; rel=back;type="image/jpeg" :rtype: list z '"z, * '"r)rr0r.rr) rlinks replace_charsvalrerlinkrrr5r5r6parse_header_links=s&   rrXrBcCs|dd}|tjtjfkr,dS|ddtjkrIdS|ddtjtjfkrodS|jt}|dkrd S|dkr|dddtkrd S|d ddtkrd S|dkr|ddt krd S|d dt krdSdS)z :rtype: str Nzutf-32rz utf-8-sigrBzutf-16rzutf-8z utf-16-berz utf-16-lez utf-32-bez utf-32-le) r BOM_UTF32_LE BOM_UTF32_BEBOM_UTF8 BOM_UTF16_LE BOM_UTF16_BEr_null_null2_null3)datasample nullcountr5r5r6guess_json_utfhs*"   rcCsPt||\}}}}}}|s4||}}t||||||fS)zGiven a URL that may or may not have a scheme, prepend the given scheme. Does not replace a present scheme with the one provided as an argument. :rtype: str )r r)re new_schemerrbr\rqueryfragmentr5r5r6prepend_scheme_if_neededs! r#c CsSt|}y"t|jt|jf}Wnttfk rNd}YnX|S)z{Given a url with authentication components, extract them into a tuple of username,password. :rtype: (str,str) r)rr)r r usernamepasswordrdr)rerauthr5r5r6get_auth_from_urls  " r's^\S[^\r\n]*$|^$z^\S[^\r\n]*$|^$c Cs|\}}t|tr$t}nt}y#|j|sLtd|Wn1tk rtd||t|fYnXdS)zVerifies that header value is a string which doesn't contain leading whitespace or return characters. This prevents unintended header injection. :param header: tuple, in the format (name, value). z7Invalid return character or leading space in header: %sz>Value for header {%s: %s} must be of type str or bytes, not %sN)r`r _CLEAN_HEADER_REGEX_BYTE_CLEAN_HEADER_REGEX_STRr1rrtype)rrmrpatr5r5r6check_header_validitys   r,cCsct|\}}}}}}|s1||}}|jddd}t|||||dfS)zW Given a url remove the fragment and the authentication part. :rtype: str @rrrp)r rsplitr)rerrbr\rr!r"r5r5r6 urldefragauths  r/c Cs~t|jdd}|dk rnt|jtrny||jWqzttfk rjtdYqzXn tddS)zfMove file pointer back to its recorded starting position so it can be read again on redirect. rANz;An error occurred when rewinding request body for redirect.z+Unable to rewind request body for redirect.)rqbodyr`_body_positionrrKr-r)prepared_request body_seekr5r5r6 rewind_bodysr4)rr)b__doc__r contextlibrCrEr0rrsysr{rIrw collectionsrrrr_internal_utilsrcompatrrrr r r r r rrrrrrrrcookiesr structuresr exceptionsrrrrr[whereDEFAULT_CA_BUNDLE_PATHZ DEFAULT_PORTSplatformr7r;rPrlrtrrrrrrrrrrrrrr frozensetrrrrrrrcontextmanagerrrrrrr rencoderrrrr#r'rr(r)r,r/r4r5r5r5r6 s           ^"  ' =9    #      %      =  &