adamwen829.github.io

Darkof

之前从来没觉得 offset 有什么坑，也没有细想过 mysql 的 offset 的实现原理。直到这周打算把 4000w 的数据热到 redis 中，写了一个脚本, 主要的代码大概如下. Table # Mysql Table Object. Redis cli = Redis(xxxxxxxxxxx). Query = text( " "SELECT id FROM example. LIMIT CUR, 1000. ORDER BY id DESC" "). Result = table.execute(query).fetchall(). Pipe = redis cli.pipeline() # 使用 pipeline 来减少连接开销. Pipeset(item.id, 'foo'. 上午 11 点左右来公司发现，才完成了 1000w 左右的数据，内心是崩溃的。找到了原因重写了一把脚本，把 offset 改成 where 就解决了这个问题，然后用了半个小时就跑完了数据- -。 Python 中的全局变量的小细节. Name = ' main '. 同理，在 main.py 中修改.

http://adamwen829.github.io/

OVERVIEW OF adamwen829.github.io

TRAFFIC RANK

>1,000,000

REVIEWS

0

PAGES IN THIS WEBSITE

18

LINKS TO THIS WEBSITE

CONTACTS

ADDRESSES

SOCIAL LINKS

ONLINE SINCE

WEBSITE DETAILS

SEO

PAGES

SIMILAR SITES

TRAFFIC RANK FOR ADAMWEN829.GITHUB.IO

TODAY'S RATING

>1,000,000

TRAFFIC RANK - AVERAGE PER MONTH

BEST MONTH

December

AVERAGE PER DAY Of THE WEEK

HIGHEST TRAFFIC ON

Saturday

TRAFFIC BY CITY

Sign up

CUSTOMER REVIEWS

Average Rating: 4.2 out of 5 with 14 reviews

5 star

6

4 star

5

3 star

3

2 star

0

1 star

0

Hey there! Start your review of adamwen829.github.io

AVERAGE USER RATING

Write a Review

WEBSITE PREVIEW

LOAD TIME

0.8 seconds

CONTACTS AT ADAMWEN829.GITHUB.IO

ADD CONTACT

Login

TO VIEW CONTACTS

Remove Contacts

FOR PRIVACY ISSUES

CONTENT

PAGES IN
THIS WEBSITE

18

SSL

EXTERNAL LINKS

1

SITE IP

151.101.192.133

LOAD TIME

0.813 sec

SCORE

6.2

PAGE TITLE

Darkof | adamwen829.github.io Reviews

<META> DESCRIPTION

之前从来没觉得 offset 有什么坑，也没有细想过 mysql 的 offset 的实现原理。直到这周打算把 4000w 的数据热到 redis 中，写了一个脚本, 主要的代码大概如下. Table # Mysql Table Object. Redis cli = Redis(xxxxxxxxxxx). Query = text( SELECT id FROM example. LIMIT CUR, 1000. ORDER BY id DESC ). Result = table.execute(query).fetchall(). Pipe = redis cli.pipeline() # 使用 pipeline 来减少连接开销. Pipeset(item.id, 'foo'. 上午 11 点左右来公司发现，才完成了 1000w 左右的数据，内心是崩溃的。找到了原因重写了一把脚本，把 offset 改成 where 就解决了这个问题，然后用了半个小时就跑完了数据- -。 Python 中的全局变量的小细节. Name = ' main '. 同理，在 main.py 中修改.

<META> KEYWORDS

1 darkof

2 adam wens blog

3 archives

4 twitter

5 github

6 mysql offset 为什么这么慢

7 from

8 redis import

9 redis

10 sqlalchemy import

CONTENT

Page content here

KEYWORDS ON PAGE

darkof,adam wens blog,archives,twitter,github,mysql offset 为什么这么慢,from,redis import,redis,sqlalchemy import,select,import,cur = 0,while,cur = max,item in,result,pipeexecute,开始执行大概下午 6 点左右，然后我就去吃饭逗猫写代码又睡了一觉,thumbnail,mysql,file foo py,global,file main py

SERVER

GitHub.com

CONTENT-TYPE

utf-8

GOOGLE PREVIEW

Darkof | adamwen829.github.io Reviews

https://adamwen829.github.io

之前从来没觉得 offset 有什么坑，也没有细想过 mysql 的 offset 的实现原理。直到这周打算把 4000w 的数据热到 redis 中，写了一个脚本, 主要的代码大概如下. Table # Mysql Table Object. Redis cli = Redis(xxxxxxxxxxx). Query = text( " "SELECT id FROM example. LIMIT CUR, 1000. ORDER BY id DESC" "). Result = table.execute(query).fetchall(). Pipe = redis cli.pipeline() # 使用 pipeline 来减少连接开销. Pipeset(item.id, 'foo'. 上午 11 点左右来公司发现，才完成了 1000w 左右的数据，内心是崩溃的。找到了原因重写了一把脚本，把 offset 改成 where 就解决了这个问题，然后用了半个小时就跑完了数据- -。 Python 中的全局变量的小细节. Name = ' main '. 同理，在 main.py 中修改.

INTERNAL PAGES

adamwen829.github.io

1

Category: 数据库 | Darkof

http://adamwen829.github.io/categories/Linux/数据库

Wget http:/ fastdl.mongodb.org/linux/mongodb-linux-i686-2.2.3.tgz. Tar -zxvf mongodb-linux-i686-2.2.3.tgz. Sudo mv mongodb-linux-i686-2.2.3.tgz /usr/local/mongodb. Sudo useradd -r mongodb. Sudo chown mongodb /var/lib/mongodb. 配置文件地址 https:/ github.com/adamwen829/mongodb-conf. Git clone git:/ github.com/adamwen829/mongodb-conf.git. Sudo cp mongodb /etc/init.d/. Sudo chmod x /etc/init.d/mongodb. Sudo cp mongodb.conf /etc/. Sudo /etc/init.d/mongodb start. Sudo /etc/init.d/mongodb stop. Python 中的全局变量的小细节.

2

Category: code snippet | Darkof

http://adamwen829.github.io/categories/code-snippet

在django文档中没有找到, 从stackoverflow查到了一个不错的答案. From django.test.client import Client. 方法1: 构造实例的时候传入headers中的key value, 以后请求方法自动传递headers. Client = HttpClient(REMOTE ADDR=’127.0.0.1’). Client.get(‘/path/‘, {‘param’:’foo’},. HTTP USER AGENT’:’firefox-22’,. REMOTE ADDR’:’127.0.0.1’}). Python 中的全局变量的小细节.

3

Tag: aws | Darkof

http://adamwen829.github.io/tags/aws

之前就遇到过一次, 今天又有同事遇到, 总之 boto 是个很神奇的项目, 超多 issue 而且跟着文档走 S3 基本不可用, 经常会遇到 400 错误 T T. 所以单独记录下. 首先在目录下创建 .boto 文件, 写入 access key 和 secret key:. Aws access key id = YOURID. Aws secret access key = YOURKEY. 然后连接的时候要注意设置 validate 为 False:. C = boto.s3.connect to region(Location.CNNorth1). Cget bucket(BUCKET NAME, validate= False. Python 中的全局变量的小细节.

4

Tag: 运营商缓存 | Darkof

http://adamwen829.github.io/tags/运营商缓存

很久之前遇到的一个问题, 趁还记得, 记录下来, 避免日后忘记. 在公司上线 2014特别项目之后, 有用户反馈出现了穿好问题. 这个这问题之前从来没有出现过. 通过检查服务器端未出现 Session 冲突, 用户在主站访问正常, 但是在进入 2014 项目之后, 发现信息出现串号现象. 所有用户都会请求这个地址, 然后由服务器动态生成 Response 并返回, 通过在线上检查发现返回的数据也没有错误. 后来又发现, 遇到的串号用户基本上集中在相同的几个地区. 通过这个线索, 发现问题是出在 ISP服务商的环节, 有些小的运营商会对你的静态数据做 cache 来加快用户访问速度, 即使你加了 no-cache 的设置, 但是仍然会有些运营商不按照规范强行缓存. 利用 UNIX时间戳来避开运营商缓存的问题, 最终问题得以解决. 作为总结, 在以后设计 API 的时候尽量要将每个用户的请求地址设计为独一无二的地址, 避免服务器对同一地址动态生成不同的数据, 最终避开这个问题. Python 中的全局变量的小细节.

5

Tag: Python | Darkof

http://adamwen829.github.io/tags/python

在周六参加 TDD Workshop 的时候, 遇到一个问题就是因为涉及到浮点数运算导致单元测试迟迟通不过- -. 回来就在这方面查了查. 不管是什么数, 在计算机中最终都会被转化为 0 和 1 进行存储, 所以我们需要先弄明白以下几点问题. 小数部分乘 2 取整数部分, 然后顺序排列. 整数部分的二进制表示为 10, 小数部分我们逐步来算. 025 * 2 = 0.5 整数部分取 0. 05 * 2 = 1.0 整数部分取 1. 所以 225 的二进制表示为 10.01. 01 * 2 = 0.2 整数部分取 0. 02 * 2 = 0.4 整数部分取 0. 04 * 2 = 0.8 整数部分取 0. 08 * 2 = 1.6 整数部分取 1. 06 * 2 = 1.2 整数部分取 1. 02 * 2 = 0.4 整数部分取 0. 所以你会发现, 0.1 的二进制表示是 0.000110011001100110011001100110011. 这就引出了一个问题, 你永远不能存下 0.1 的二进制, 即使你把全世界的硬盘都放在一起, 也存不下 0.1 的二进制小数. 剩下的 52 bit 用来存储尾数.

UPGRADE TO PREMIUM TO VIEW 13 MORE

TOTAL PAGES IN THIS WEBSITE

18

SOCIAL ENGAGEMENT

wenxiaobin

OTHER SITES

adamwelz.net

Adam Welz's Site - About Me

I'm a South African writer, photographer and filmmaker with particular experience of environmental issues. I have a couple of biology degrees and have lived and worked on four continents. I'm currently based in New York, USA, where I produce media and do research for clients around the world. You can see examples of my work and connect with me by clicking through the tabs below. Please use the contact form to request an updated CV.

adamwelz.org

adamwelz.wordpress.com

Adam Welz's Weblog | enviro stuff, birds, random bits n pieces

Skip to search - Accesskey = s. Adam Welz's Weblog. Posted in ALL BLOG POSTS. By adamwelz on July 29, 2009. Welcome to my weblog. Where I intermittently post bits and pieces of my work, some casual, some professional, as well as odd things I find like an early Robert Mugabe speech. Menu on the top right of this page to pull out what you want to see, or just start reading down from here, where you’ll find the most recently-added (unsorted) posts. My wildlife conservation blog on The Guardian is now live.

adamwelznxcam.wordpress.com

Adam Welz's NXCAM Blog | Just another WordPress.com weblog

Adam Welz's NXCAM Blog. Just another WordPress.com weblog. About this NXCAM blog. Best color / profile settings for gradeable video – Sony NEX-5N. October 17, 2011. I’ve been shooting a little more with my NEX-5N, also reading a little more, especially about color settings to retain detail across the maximum brightness range (i.e. squeeze maximum latitude out of the camera) while minimising artifacts to deliver the most gradeable footage out of some pretty difficult, harshly-lit situations. In summary: F...

adamwen.com

www.adamwen.com

This Web page parked FREE courtesy of Domains Priced Right. Search for domains similar to. Is this your domain? Let's turn it into a website! Would you like to buy this. Find Your Own Domain Name. See our full line of products. Easily Build Your Professional Website. As low as $4.99/mo. Call us any time day or night (480) 624-2500.

adamwen829.github.io

Darkof

之前从来没觉得 offset 有什么坑，也没有细想过 mysql 的 offset 的实现原理。直到这周打算把 4000w 的数据热到 redis 中，写了一个脚本, 主要的代码大概如下. Table # Mysql Table Object. Redis cli = Redis(xxxxxxxxxxx). Query = text( " "SELECT id FROM example. LIMIT CUR, 1000. ORDER BY id DESC" "). Result = table.execute(query).fetchall(). Pipe = redis cli.pipeline() # 使用 pipeline 来减少连接开销. Pipeset(item.id, 'foo'. 上午 11 点左右来公司发现，才完成了 1000w 左右的数据，内心是崩溃的。找到了原因重写了一把脚本，把 offset 改成 where 就解决了这个问题，然后用了半个小时就跑完了数据- -。 Python 中的全局变量的小细节. Name = ' main '. 同理，在 main.py 中修改.

adamwendel.com

Adam Wendel

Designer and strategist dedicated to creating truly relevant and compelling solutions that improve and enrich people’s lives.

adamwenger.com

adam wenger

Writer. Editor. Producer. Social Media Manager. According to the Red Cross. About 9.2 million people in the U.S. donate blood each year, leading to approximately 15.7 million donations. Ever wondered where all that blood goes? The history of the flu shot, from epidemic to vaccines to celebrity paranoia. The first case of HIV was reported in 1981. Today, more than 1 million Americans live with the virus. There is no cure. John Cho dishes on. The Very Best Hair of World Cup 2014. Stunning goals, improbable...

adamwenner.com

This Web site coming soon

If you are the owner of this web site you have not uploaded (or incorrectly uploaded) your web site. For information on uploading your web site using FTP client software or web design software, click here for FTP Upload Information.

adamwennick.com

Adam Wennick- Actor/Voiceover Artist

Shakespearean voice with realism training. The official website of Actor/Voiceover Artist Adam Wennick. Adam Wennick- Actor/Voiceover Artist. Shakespearean voice with realism training. Actor/Director Adam Wennick's homepage.

adamwentworth860fall2015.wordpress.com

Adam Wentworth, 860 FALL 2015

Adam Wentworth, 860 FALL 2015. Initial Design Package: Design. Initial Design Package: Research. Initial Design Package: Concept. Artistic Influence from the 21st Century. Research into Monarch Butterflies. Initial Design Package: Design. September 10, 2015. Initial Design Package: Design. Initial Design Package: Research. September 10, 2015. Artists From Three Centuries. A Sunday Afternoon on the Island of La Grande Jatte. Seven Samurai ( http:/ www.imdb.com/title/tt0047478/. September 10, 2015. Young M...