Posted in
Windows Powershell |
1 Comment | 5,190 views | 24/10/2009 15:19
Could you imagine what can you do with Powershell? I imagined to create a search engine with Powershell. I used Google for Search and filtered results. Looking codes would be helpful you to have an idea about Google Parsing.
I search for Dressy Bessy – New Song:
Did it match with Keywords?
You can see my Powershell script. Default search is a Turkish Music Group “Gece” and their Album “Icinde Sakli”. You can change default search parameters from script. Also adding more Google page would be increase accurate.
Preview of Script:
####################################
# Google Parser + Mp3 Searcher #
# Coded By Yusuf Ozturk #
# http://www.yusufozturk.info #
####################################
[void][System.Reflection.Assembly]::LoadWithPartialName('Microsoft.VisualBasic')
# Search Input
$SearchInput = [Microsoft.VisualBasic.Interaction]::InputBox("Search", "Song", " ")
# Split to Keywords
$Keyword = $SearchInput -Replace "\s","+" # For searching
$Keyword = $Keyword + "+rapidshare.com+mp3" # Look for Rapidshare Links
$Keywords = $Keyword.Split("+") # For checking purposes
# Where Could We Write?
$Location = "C:\Search"
# Lets Check Location Directory
If ((Test-Path -Path $Location) -ne $True)
{
New-Item $Location -Type Directory
}
# Search Outputs
$Search = "$Location\Search.txt"
$Results = "$Location\Results.txt"
$Temp = "$Location\Temp.txt"
# Lets Make Google Search
$Url = "http://www.google.com.tr/search?hl=tr&source=hp&q=" + $Keyword + "&start=0"
$Webclient = New-Object System.Net.WebClient
$Webpage = $Webclient.DownloadString($Url)
$Webpage > "$Search"
$Url = "http://www.google.com.tr/search?hl=tr&source=hp&q=" + $Keyword + "&start=10"
$Webclient = New-Object System.Net.WebClient
$Webpage = $Webclient.DownloadString($Url)
$Webpage >> "$Search"
# Get Web Results and Parse Web Links
$Hostname = Select-String "$Search" -Pattern 'http://'
$Hostname = [Regex]::Matches($Hostname, '<a class="l')" href="([^">> "$Location\$ID.txt" |
####################################
# Google Parser + Mp3 Searcher #
# Coded By Yusuf Ozturk #
# http://www.yusufozturk.info #
####################################
[void][System.Reflection.Assembly]::LoadWithPartialName('Microsoft.VisualBasic')
# Search Input
$SearchInput = [Microsoft.VisualBasic.Interaction]::InputBox("Search", "Song", " ")
# Split to Keywords
$Keyword = $SearchInput -Replace "\s","+" # For searching
$Keyword = $Keyword + "+rapidshare.com+mp3" # Look for Rapidshare Links
$Keywords = $Keyword.Split("+") # For checking purposes
# Where Could We Write?
$Location = "C:\Search"
# Lets Check Location Directory
If ((Test-Path -Path $Location) -ne $True)
{
New-Item $Location -Type Directory
}
# Search Outputs
$Search = "$Location\Search.txt"
$Results = "$Location\Results.txt"
$Temp = "$Location\Temp.txt"
# Lets Make Google Search
$Url = "http://www.google.com.tr/search?hl=tr&source=hp&q=" + $Keyword + "&start=0"
$Webclient = New-Object System.Net.WebClient
$Webpage = $Webclient.DownloadString($Url)
$Webpage > "$Search"
$Url = "http://www.google.com.tr/search?hl=tr&source=hp&q=" + $Keyword + "&start=10"
$Webclient = New-Object System.Net.WebClient
$Webpage = $Webclient.DownloadString($Url)
$Webpage >> "$Search"
# Get Web Results and Parse Web Links
$Hostname = Select-String "$Search" -Pattern 'http://'
$Hostname = [Regex]::Matches($Hostname, '<a class="l')" href="([^">> "$Location\$ID.txt"
Download Google Parser Script
I used Google.com.tr to Search Mp3. I hope Google will get banned by ASCAP :D
As always: Go to Hell Google!
Please use Bing for a better future.
Posted in
Windows Powershell |
No Comment | 2,739 views | 22/09/2009 02:10
Bugün bu sorunun cevabını aramak için oturdum Powershell ISE karşısına. Bildiğiniz gibi Windows 7 ile birlikte Powershell ISE yüklü olarak geliyor ve Powershell üzerinde uygulama yapabilmenize olanak sağlıyor. Peki neden Powershell ile MP3 arama motoru yapmak istediğimi biliyor musunuz? Hemen söyleyeyim, Mü-Yap yüzünden.
Bilmiyorum farkında mısınız ama bugün Myspace ve Last.FM, Mü-Yap tarafından açılmış bir dava nedeniyle Türkiye içinden erişime kapatıldı. Ben de bu yüzden insanların MP3’leri daha kolay bulabilmesi ve erişebilmesi için bu arama motorunu yaptım. Tüm sonuçlar Google arama motoru üzerinden bulunarak size sunuluyor. Eğer Mü-Yap insanların MP3’lere kolay ulaşmalarını engellemek istiyorsa, buyursun Google’ı kapatsın!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
| $icerik = "umut+kaya+mevsimler+gecerken+rapidshare.com+mp3"
$parcala = $icerik.Split("+")
$search = "C:\Search\Search.txt"
$results = "C:\Search\Results.txt"
$temp = "C:\Search\Temp.txt"
$url = "http://www.google.com.tr/search?hl=tr&source=hp&q=" + $icerik + "&start=0"
$webclient = new-object System.Net.WebClient
$webpage = $webclient.DownloadString($url)
$webpage > "$search"
$url = "http://www.google.com.tr/search?hl=tr&source=hp&q=" + $icerik + "&start=10"
$webclient = new-object System.Net.WebClient
$webpage = $webclient.DownloadString($url)
$webpage >> "$search"
$url = "http://www.google.com.tr/search?hl=tr&source=hp&q=" + $icerik + "&start=20"
$webclient = new-object System.Net.WebClient
$webpage = $webclient.DownloadString($url)
$webpage >> "$search"
$Hostname = Select-String "$search" -pattern 'http://'
$Hostname = [regex]::matches($Hostname, '<a href="([^"]+)" class=l')
$id="0";
foreach ($i in $Hostname)
{
$i = [string]$i
$i = $i.Substring(9)
$i = [string]$i[0..($i.length-10)] -replace "\s",""
write-host $i
$id=[int]$id+1;
$url = "$i"
$webclient = new-object System.Net.WebClient
$webpage = $webclient.DownloadString($url)
$webpage >> "C:\Search\$id.txt"
$Status = "True"
foreach ($b in $parcala)
{
$Kontrol = Select-String "C:\Search\$id.txt" -pattern "$b"
If(!$Kontrol)
{
$Status = "False"
write-host False aldik burada
}
}
If ($Status -eq "False")
{
write-host "Eslesme yok"
}
Else
{
write-host "Eslesme var"
$webpage >> "$results"
}
}
$LinkKontrol = Select-String "$results" -pattern 'http://'
$LinkKontrol = [regex]::matches($LinkKontrol, 'href="http://rapidshare.com/files/([^"]+)"')
$rapidid="0";
foreach ($c in $LinkKontrol)
{
$c = [string]$c
$c = $c.Substring(6)
$c = [string]$c[0..($c.length-2)] -replace "\s",""
write-host $c
#Link Checker
$c = $c.Substring(7)
$files = $c.Split("/")[2]
$filenames = $c.Split("/")[3]
$url = "http://api.rapidshare.com/cgi-bin/rsapi.cgi?sub=checkfiles_v1&files=" + $files + "&filenames=" + $filenames
$webpage = $webclient.DownloadString($url)
$rapidid=[int]$rapidid+1;
$linkcheck = "C:\Search\rapid-$rapidid.txt"
$webpage > "$linkcheck"
$sonuclink = Select-String "$linkcheck" -pattern "$files"
$sonuclink = $sonuclink.get_Line()
$Status = $sonuclink.Split(",")[4]
If ($Status -eq "1")
{
add-content -path “C:\Sonuclar.txt” -value $c
}
} |
$icerik = "umut+kaya+mevsimler+gecerken+rapidshare.com+mp3"
$parcala = $icerik.Split("+")
$search = "C:\Search\Search.txt"
$results = "C:\Search\Results.txt"
$temp = "C:\Search\Temp.txt"
$url = "http://www.google.com.tr/search?hl=tr&source=hp&q=" + $icerik + "&start=0"
$webclient = new-object System.Net.WebClient
$webpage = $webclient.DownloadString($url)
$webpage > "$search"
$url = "http://www.google.com.tr/search?hl=tr&source=hp&q=" + $icerik + "&start=10"
$webclient = new-object System.Net.WebClient
$webpage = $webclient.DownloadString($url)
$webpage >> "$search"
$url = "http://www.google.com.tr/search?hl=tr&source=hp&q=" + $icerik + "&start=20"
$webclient = new-object System.Net.WebClient
$webpage = $webclient.DownloadString($url)
$webpage >> "$search"
$Hostname = Select-String "$search" -pattern 'http://'
$Hostname = [regex]::matches($Hostname, '<a href="([^"]+)" class=l')
$id="0";
foreach ($i in $Hostname)
{
$i = [string]$i
$i = $i.Substring(9)
$i = [string]$i[0..($i.length-10)] -replace "\s",""
write-host $i
$id=[int]$id+1;
$url = "$i"
$webclient = new-object System.Net.WebClient
$webpage = $webclient.DownloadString($url)
$webpage >> "C:\Search\$id.txt"
$Status = "True"
foreach ($b in $parcala)
{
$Kontrol = Select-String "C:\Search\$id.txt" -pattern "$b"
If(!$Kontrol)
{
$Status = "False"
write-host False aldik burada
}
}
If ($Status -eq "False")
{
write-host "Eslesme yok"
}
Else
{
write-host "Eslesme var"
$webpage >> "$results"
}
}
$LinkKontrol = Select-String "$results" -pattern 'http://'
$LinkKontrol = [regex]::matches($LinkKontrol, 'href="http://rapidshare.com/files/([^"]+)"')
$rapidid="0";
foreach ($c in $LinkKontrol)
{
$c = [string]$c
$c = $c.Substring(6)
$c = [string]$c[0..($c.length-2)] -replace "\s",""
write-host $c
#Link Checker
$c = $c.Substring(7)
$files = $c.Split("/")[2]
$filenames = $c.Split("/")[3]
$url = "http://api.rapidshare.com/cgi-bin/rsapi.cgi?sub=checkfiles_v1&files=" + $files + "&filenames=" + $filenames
$webpage = $webclient.DownloadString($url)
$rapidid=[int]$rapidid+1;
$linkcheck = "C:\Search\rapid-$rapidid.txt"
$webpage > "$linkcheck"
$sonuclink = Select-String "$linkcheck" -pattern "$files"
$sonuclink = $sonuclink.get_Line()
$Status = $sonuclink.Split(",")[4]
If ($Status -eq "1")
{
add-content -path “C:\Sonuclar.txt” -value $c
}
}
Görebileceğiniz gibi en üstte arama yapmak istediğim kelimeleri, aralarında + olacak şekilde giriyorum. Arama kelimeleri içine rapidshare.com ve mp3 yazmayı unutmuyorum. Böylece sadece rapidshare linklerine sahip sitelerin arama sonucu olarak çıkmasını sağlıyoruz. Dikkat etmeniz gereken, C:\ içersinde Search isimli bir klasör yaratmış olmanız ve C:’ye erişim yetkinizin olması. Her arama sonrası, yaratılan .txt içersini boşaltmanızda fayda var. Ayrıca dikkat ettiyseniz, çıkan sonuçları Rapidshare API’si kullanarak, aktif olup olmadığını kontrol ederek, Sonuclar.txt içersine atıyorum. Böylece sonuçlar arasında ölü link bulunması gibi bir ihtimal olmuyor.