Pages

Showing posts with label Python. Show all posts
Showing posts with label Python. Show all posts

Sunday, October 19, 2008

Fetch urls anonymously with Python urllib2 and Tor

For various purposes (web scrapping, spiders, data extraction, etc) you need to use anonimity... 1. Install Tor 2. Check if Tor is working 3. Write your script in Python
import urllib2
proxy_support = urllib2.ProxyHandler({"http":"http://127.0.0.1:8118"})
opener = urllib2.build_opener(proxy_support)
url='http://whatismyip.com/'
page = opener.open(url)
contents=page.read()
print contents

And that is all

For web scrapping you can use Beautiful Soup. Adding some Beautiful Soup:
import re
from BeautifulSoup import BeautifulSoup
soup = BeautifulSoup(page)
h1Tags = soup.findAll('h1')
#ip address text is in a 2nd h1 tag:
ip = re.sub(r'<[^>]*?>', '', str(h1Tag[1]))
print ip
If you want to see some readable text, see the page source, there is a comment about which url you can access to see only the IP address...

Tuesday, July 11, 2006

Creole Trac - Ticket spam

Well, spammers get better, after blog spamming, trac is vulnerable. Creole is using trac for development, and while browsing through tickets, found this: Public comments and new tickets seem to be suppresed now for anonymus users. Example: Ticket: MSSQL driver errors Edgewall is working on a spam filter, which will be released with 0.10 Milestone Until then, disable anonymus content or you can try experimental spam filter. A quick fix could be:
DELETE FROM ticket_change WHERE author = "spammer_name";
--update  ticket version and milestone, which is altered
UPDATE ticket SET version = '', milestone = '' WHERE version = 'HEAD' AND milestone = 'MILESTONE_NAME';

Tuesday, May 24, 2005

PySerial

A really nice module for Python that proved efficient and easy to use if pySerial... There are many devices that can comunicate with a PC through a serial port, so here is a script(listener) that uses pySerial to catch the output from such a device and store it in a file. Can be ajusted to handle a faster output from serial port...
import serial

#open second serial port, 0 for the first
ser=serial.Serial(1)

#loop forever
while 1:
  s=ser.readline()
  fileLog=open('serial_log.txt','a')
  fileLog.write('%s'%s)
  fileLog.close()
ser.close()