一、背景
目标链接:/default.aspx?tabid=226 ,中国土地市场网,获取相应的行政区代码、标题、详情页链接和发布时间
二、过程
1、确定加密参数
抓包,数据就在该链接中,复制url和请求头,请求,没有返回数据,返回了一段js代码
function stringToHex(str) {var val = "";for (var i = 0; i < str.length; i++) {if (val == "") val = str.charCodeAt(i).toString(16);else val += str.charCodeAt(i).toString(16);}return val;}function YunSuoAutoJump() {var width = screen.width;var height = screen.height;var screendate = width + "," + height;var curlocation = window.location.href;if ( - 1 == curlocation.indexOf("security_verify_")) {document.cookie = "srcurl=" + stringToHex(window.location.href) + ";path=/;";}self.location = "/default.aspx?tabid=226&security_verify_data=" + stringToHex(screendate);}
暂时先不管它,清空cookie,浏览器设置代理,用charles进行抓包,发现总共请求了三次才获得真实数据
第一次请求,增加了cookie值yunsuo_session_verify,同时返回js代码,该代码生成了一个cookie(srcurl)和url
第二次请求,携带cookie(yunsuo_session_verify和srcurl),请求了js生成的url,并新增了cookie(security_session_mid_verify)
第三次请求,携带生成的cookie,请求真实链接,正常返回数据
OK,整体流程已经明白了,下面看看js代码
2、处理js函数
screendate, 就是屏幕的宽高,可以自己写死
url和cookie的生成,都是通过stringToHex函数,对屏幕宽高和请求链接进行了加密
stringToHex函数,遍历字符串的每个字符,对字符进行转换为Unicode编码,然后将编码转为16进制,在拼接起来
def str2token(str_data):token = ''for val in str_data:token += str(hex(ord(val)))return token.replace('0x', '')def china_land(url):screen_data = "1333,800"cookie = ''if "security_verify_" not in url:cookie = 'srcurl=' + str2token(url) + ';path=/;'url = "/default.aspx?tabid=226&security_verify_data=" + str2token(screen_data)return url, cookie
在转16进制时,有个小坑,一开始以为和Python的十六进制转换一样,但是发现生成的字符串和实际的不一样,然后查了javascript的16进制转换语法,发现转换后的数据比python的少了“0x”,因此要将‘0x’去除
from public_method import *url = "/default.aspx?tabid=226"session = requests.session()res = session.get(url)verify_url, cookie = china_land(url)print(verify_url, cookie)res = session.get(verify_url)header = {"Host": "","User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:65.0) Gecko/0101 Firefox/65.0","Accept": "text/css,*/*;q=0.1","Accept-Language": "zh-CN,zh;q=0.8,zh-TW;q=0.7,zh-HK;q=0.5,en-US;q=0.3,en;q=0.2","Accept-Encoding": "gzip, deflate","Referer": url,"Connection": "keep-alive","Cookies": cookie,}response = session.get(url, headers=header)print(response.text)
另一个坑是cookie,用了reuqests库的session对象,在最后一次请求时,在headers中携带js生成的cookie,用的Cookie字段,但是没有返回正常数据,然后换了Cookies字段,就正常了,打印了两次的请求头
用cookie:{'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:65.0) Gecko/0101 Firefox/65.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'text/css,*/*;q=0.1', 'Connection': 'keep-alive', 'Host': '', 'Accept-Language': 'zh-CN,zh;q=0.8,zh-TW;q=0.7,zh-HK;q=0.5,en-US;q=0.3,en;q=0.2', 'Referer': '/default.aspx?tabid=226', 'Cookie': 'srcurl=687474703a2f2f7777772e6c616e646368696e612e636f6d2f64656661756c742e617370783f74616269643d323236;path=/;'}用cookies: {'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:65.0) Gecko/0101 Firefox/65.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'text/css,*/*;q=0.1', 'Connection': 'keep-alive', 'Host': '', 'Accept-Language': 'zh-CN,zh;q=0.8,zh-TW;q=0.7,zh-HK;q=0.5,en-US;q=0.3,en;q=0.2', 'Referer': '/default.aspx?tabid=226', 'Cookies': 'srcurl=687474703a2f2f7777772e6c616e646368696e612e636f6d2f64656661756c742e617370783f74616269643d323236;path=/;', 'Cookie': 'security_session_mid_verify=ce4213c94b70552197bf3caad9ac97da; yunsuo_session_verify=1a27da8d25711fad325811a16a2a4307'}
对比之后,发现如果用Cookie字段,请求头中只有js生成的cookie值,之前请求携带的cookie值就会被替换了,而这次请求时需要三个cookie值,所以需要用cookies字段,从cookie和cookies字段中,requests库都会进行cookie数据的提取,当然,如果用cookie={‘srcurl’:’…’}来携带js加密的cookie值,也是可以获取数据的
三、总结
如果觉得《中国土地市场网-js解密》对你有帮助,请点赞、收藏,并留下你的观点哦!