Hi there,

 

Before I start, thank you for reading this thread. Recently, I've been trying to code a script which will grab coding of a certain web page (not all of the web page's coding, only a specific part). After grabbing the code, I'd like the code to be used on my web page.

 

What I'm trying to do is create a search function for an online game's high score. The game's high score can be found here: http://sfo.gotdns.com/SFXScores/index.html. I'm going to make it so that once a person types in their user name within the search function, it displays their user name with kills/deaths etc. I'm going to attempt to get this working by creating a HTML form, where the user types in a specific user name and the form then submits it's data to another script; the grabber page.

 

Usually, the grabber page should have a field where it states where to grab from, and where to end the grab. E.G.:

 

$config['start_tag'] = "<table>";
$config['end_tag']   = "</table>";

 

Once a user types in their user name in the HTML form, I'm going to attempt to get the submitted data in the start tag config. So, something like:

 

$config['start_tag'] = "<?php echo $_POST["username"]; ?>";
$config['end_tag']   = "</table>";

 

After this is done, it should then search the high scores page (http://sfo.gotdns.com/SFXScores/index.html) for their user name. I know that their rank may be lower, so their name may not appear on the first page, so if possible, could it possible for the script to search several different pages till the name is found? Also, the second code which I have posted above doesn't work, so the whole script isn't currently working. After grabbing the content and placing it on my web page should be enough, and nothing else should be required.

 

Could anyone possibly help me on this? And if not, could someone at least tell me which coding would be best to use? I've taken a look at cURL and a couple of php scripts.

 

Kind regards.

Hi there,

 

Before I start, thank you for reading this thread. Recently, I've been trying to code a script which will grab coding of a certain web page (not all of the web page's coding, only a specific part). After grabbing the code, I'd like the code to be used on my web page.

 

What I'm trying to do is create a search function for an online game's high score. The game's high score can be found here: http://sfo.gotdns.com/SFXScores/index.html. I'm going to make it so that once a person types in their user name within the search function, it displays their user name with kills/deaths etc. I'm going to attempt to get this working by creating a HTML form, where the user types in a specific user name and the form then submits it's data to another script; the grabber page.

 

Usually, the grabber page should have a field where it states where to grab from, and where to end the grab. E.G.:

 

$config['start_tag'] = "<table>";
$config['end_tag']   = "</table>";

 

Once a user types in their user name in the HTML form, I'm going to attempt to get the submitted data in the start tag config. So, something like:

 

$config['start_tag'] = "<?php echo $_POST["username"]; ?>";
$config['end_tag']   = "</table>";

 

After this is done, it should then search the high scores page (http://sfo.gotdns.com/SFXScores/index.html) for their user name. I know that their rank may be lower, so their name may not appear on the first page, so if possible, could it possible for the script to search several different pages till the name is found? Also, the second code which I have posted above doesn't work, so the whole script isn't currently working. After grabbing the content and placing it on my web page should be enough, and nothing else should be required.

 

Could anyone possibly help me on this? And if not, could someone at least tell me which coding would be best to use? I've taken a look at cURL and a couple of php scripts.

 

Kind regards.

 

<?php  
class Gamebattles{
function fetch(   $url, $firsttag, $secondtag   ){

	$ch = curl_init();//Initialise CURL 

	curl_setopt($ch, CURLOPT_URL, $url);//Set the url  

	curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);//We want it to return data  

	curl_setopt ($ch, CURLOPT_CONNECTTIMEOUT, 10);//Of course, we don't want your script to run forever, so set a timeout  

	$text = curl_exec($ch);//Execute and get the page

	$exploded = explode($firsttag, $text);//Make an array of the part before and after the tag  

	$result = explode($secondtag, $exploded[1]);//Grab the part after the tag 

	echo $result[0];//Echo the result


 		curl_close($ch);//Close cURL
}
}

$test = new Gamebattles();
$test->fetch('http://url.com','tag to start grab','tag to end grab');
?>

 

I used the above code to pull the results of matches from COD matches websites. This may help you, id help you more if i could but im heading out in 10, If no one hashelped you/you haven't got it sorted when i get back ill try and sort you.

You should ask if the game has a web service, or XML datasource, etc, as it would be far more efficient to query. If you are manually parsing each page then you'd need to write a function that will get the page using CURL, load it into a DOMDocument object then find it using XPath (assuming its parsable). If the username doesn't exist then repeat on the next page until found. It would be very very intensive on CPU, Memory and Bandwidth.

 

Because of this, you'd be better scraping the whole website into a database every 6 hours or something using a cron job, then it would be a lot easier and efficient to query your own database. It would also open up a lot more opportunities for advanced data manipulation, e.g graph players rank history etc.

I used the above code to pull the results of matches from COD matches websites. This may help you, id help you more if i could but im heading out in 10, If no one hashelped you/you haven't got it sorted when i get back ill try and sort you.

 

Hey Adamoverload,

 

I really do appreciate the reply as well as providing the code. I've attempted to use the code and I can say it works perfectly well; it's amazing. I've managed to make it so that it can search every single page on the high scores page (28 pages in total). However, it does take a long time (as well as uses a lot of resources) hence the reason why I will limit the pages to 10 or 15.

 

I'm currently using the script and as a thanks back, would you like a link back to your web site? Also the script isn't yet finished, so I'll add a link back once it's done.

 

You should ask if the game has a web service, or XML datasource, etc, as it would be far more efficient to query. If you are manually parsing each page then you'd need to write a function that will get the page using CURL, load it into a DOMDocument object then find it using XPath (assuming its parsable). If the username doesn't exist then repeat on the next page until found. It would be very very intensive on CPU, Memory and Bandwidth.

 

Because of this, you'd be better scraping the whole website into a database every 6 hours or something using a cron job, then it would be a lot easier and efficient to query your own database. It would also open up a lot more opportunities for advanced data manipulation, e.g graph players rank history etc.

 

Hey Jock,

 

Thanks for your reply, I do appreciate it . I'll ask the creator of the game whether it does contain a data source or anything similar, as I'm sure it will right? Seeing as it captures rank and such. I do agree that this will use a lot of CPU, Memory as well as Bandwidth so, as stated above I will limit the search for pages up to 10 or 15; possibly even less if required. The current script I'm using (provided by Adamoverload) doesn't require the HTML content to be parsed, as it basically captures the whole HTML content within the set tags, which works perfectly fine.

 

I also do like the idea of graphing player's rank history, however, I'm unsure whether I really want this now. I'd like to start it up for now and perhaps work developing on it later. None the less, I think they're all great ideas.

 

One main thing I'd like to do now is, add the search function to the script; the form. Using the script (Adamoverload's script), I'd like to add a search function where the user can basically type in what user name to start the grabbing from. E.G.:

 

$test = new Gamebattles();
$test->fetch('http://url.com','tag to start grab','tag to end grab');
?>

 

The above code is the last three (3) lines in Adam's code. I'd like to replace "tag to start grab" with whatever the user types (most likely his/her user name). I'm currently attempting to this using a form, however, it doesn't seem to work. This is what I've done so far:

 

index.php

<form action="search.php" method="post">
User Name: <input type="text" name="username" />
<input type="submit" />
</form>

 

search.php

$test = new Gamebattles();

$test->fetch('http://url.com','$username = $_REQUEST['username']','tag to end grab');

?>

RED = Changed in actual document.

 

I've attempted to put the $username = $_REQUEST['username'] code elsewhere in the page, where it's not embeded within the <? php ?> tags and it seems to work perfectly, however it doesn't when within these tags. How can I change this so that the form works?

 

Also, I've tried using <?php echo $_POST["username"]; ?> instead of $username = $_REQUEST['username'] and it still doesn't seem to work.

 

Kind regards, I totally appreciate all the help;

Hauzer.

Hey Adamoverload,

 

I really do appreciate the reply as well as providing the code. I've attempted to use the code and I can say it works perfectly well; it's amazing. I've managed to make it so that it can search every single page on the high scores page (28 pages in total). However, it does take a long time (as well as uses a lot of resources) hence the reason why I will limit the pages to 10 or 15.

 

I'm currently using the script and as a thanks back, would you like a link back to your web site? Also the script isn't yet finished, so I'll add a link back once it's done.

 

 

 

Hey Jock,

 

Thanks for your reply, I do appreciate it . I'll ask the creator of the game whether it does contain a data source or anything similar, as I'm sure it will right? Seeing as it captures rank and such. I do agree that this will use a lot of CPU, Memory as well as Bandwidth so, as stated above I will limit the search for pages up to 10 or 15; possibly even less if required. The current script I'm using (provided by Adamoverload) doesn't require the HTML content to be parsed, as it basically captures the whole HTML content within the set tags, which works perfectly fine.

 

I also do like the idea of graphing player's rank history, however, I'm unsure whether I really want this now. I'd like to start it up for now and perhaps work developing on it later. None the less, I think they're all great ideas.

 

One main thing I'd like to do now is, add the search function to the script; the form. Using the script (Adamoverload's script), I'd like to add a search function where the user can basically type in what user name to start the grabbing from. E.G.:

 

$test = new Gamebattles();
$test->fetch('http://url.com','tag to start grab','tag to end grab');
?>

 

The above code is the last three (3) lines in Adam's code. I'd like to replace "tag to start grab" with whatever the user types (most likely his/her user name). I'm currently attempting to this using a form, however, it doesn't seem to work. This is what I've done so far:

 

index.php

<form action="search.php" method="post">
User Name: <input type="text" name="username" />
<input type="submit" />
</form>

 

search.php

 

RED = Changed in actual document.

 

I've attempted to put the $username = $_REQUEST['username'] code elsewhere in the page, where it's not embeded within the <? php ?> tags and it seems to work perfectly, however it doesn't when within these tags. How can I change this so that the form works?

 

Also, I've tried using <?php echo $_POST["username"]; ?> instead of $username = $_REQUEST['username'] and it still doesn't seem to work.

 

Kind regards, I totally appreciate all the help;

Hauzer.

 

This is OOP programming so i'd make the index.php this:

<?php
include("search.php");
$test = new Gamebattles();

if( !$_POST['submit'] )
{
echo("
<form action=\"search.php\" method=\"post\">
User Name: <input type=\"text\" name=\"username\" /><br>
<input type=\"submit\" />
</form>");
}
else
{
	print $test->fetch('http://url.com', $_POST['username'], 'tag to end grab');
}
?>

 

Remove these 2 lines from search.php:

$test = new Gamebattles();
$test->fetch('http://url.com','tag to start grab','tag to end grab');

 

Enjoy, Adam.

Thanks for the replies Adam, they've really helped and couldn't have done it without you . The script is currently working and does capture the statistics of a player. One slight thing though, is it possible to disable the case-sensitive like feature, as I want users to be able to type in their name without worrying about case sensitivity. Especially those with name like "(-ObNoXiOuS-1-)".

 

No worries if not, as I do already appreciate the script. Would you like a link-back?

Not 100% sure mate, try the PHP manual, people may have scripts to do that!

 

About a link back, until i have a portfolio, No thanks

Hey Adam,

 

I really appreciate all the help you've given me! If possible, once a user types in their user name within the search form, could it replace the URL's user name section to be their user name? As well as grab the content and place it onto my web page?

 

Unfortunately, I can't use the grabber script you (Adam) provided as it requires start and end (where to grab from) tags, while the document is dynamic and is different for every person, so no tags can be used. I need to grab the whole content.

 

Again, thank you for all the responses made in this thread, I really do appreciate them all. Also, sorry for all the hassle I've caused.

Im not too sure, i rarely use CURL