Topic: vgmapper, get while it's there

[Trop]

I'll keep this simple.  Yesterday my parents were in a car wreck.  Things aren't looking good and it's more likely then not the next bill I get for vgmapper.com isn't gonna get paid.  Even though it shouldn't be for a few months the site will most likely go down so if there's something you want off it take it now.

[DarkWolf]

Sorry to hear about your parents.  Do you have a local copy of everything?  If not, I will try to make one and hold onto it.

[Trop]

Yes I'll always have a local copy of everything so you can email me if you even want something specific.

[JonLeung]

Hey, Trop.  That's really sad to hear.  Are your parents okay?

As for the maps, I'll be happy to keep your work alive and by having them available here on VGMaps if I don't already have them, if you're absolutely sure the end of your site is imminent.  I'm sure you already know that you and any mappers will always be welcome here on VGMaps.

It hasn't happened much, simply because there haven't been many other screenshot map-focussed web sites, but I've always been sad when those sites stop updating, or go offline altogether.  At first I might have been happy about less "competition", but there's usually an unfortunate reason why they have to end.  Not that I mean to trivialize the accident, of course.  We all have our "real-life" priorities, and web sites are rarely that high on anyone's list, especially when stuff like this happens.

I'm sure that I, and the VGMaps community, will keep your family in our thoughts and prayers.  If/when you get a chance, hopefully you will be able to update us on the situation.

[Trop]

Well my dad is gone and it looks like my mom will need a lot of special care for the rest of her life.  Don't really have time to think right now since I have so many things to rearrange.  It always seems like the most hell falls on you when you're in the worst emotional state.

Yes, about the site, let me give you the run down.
Any map ending with an _02 is one of mine.  TAKE THEM ALL Jon.  Don't take you're time, there's a lot.
Any _03 is by RayVGM of VGMuseum, you probably already have all his stuff.
Any _04 is by KungFuFurby from SNESMusic.com, you can probably find his email there if you want his stuff.
You can ignore _05.  Kinsuke JP only ever made Japanese to English text images and they are all up at GameFAQS anyway.
There are no _06's or _07's Alex and Jessica were only honorary contributors so no maps.
_08, Radien from Purezc.com, merely remade some of my ZeldaTP Wii maps into ZeldaTP Game Cube maps.  It wasn't much but he was very enthusiastic and it would be nice if you posted them here.
_09, Star Fighters 76, already a contributor here of course, but lately he wants you to get his maps off GameFAQs yourself so I may have a few you don't.  Of course you could just go to GameFAQS.
DarkWolf is _10 so nothing for you to do there.
FlyingArmor is _11 so again you already have it all.
_12 is Marsil Marsil and I only ever got one map in one email from him but I'm sure he'd want his work saved so you should probably just take it.
_13 is Peardain.
_14 is Paco.
Both are already covered here and that's it.

[TerraEsperZ]

Oh man I'm so sorry for your parents  . I know there's not much we can do to help, but given that you mentioned money being a problem in the near future, maybe we could donate you some money? I know that probably sounds materialistic like hell at the moment, but money problems don't seem to care about things like life and death ...

[|N|NjA|]

Indeed, sorry about your parents... must be a hell of a situation right now :/

[DarkWolf]

Jon, you might find this useful.  I crawled the website and made a CSV of image files that followed the format Trop described.

http://garoux.net/forum/vgmapper_images.csv

[Trop]

Thank you for your concern Terra but I should have enough to keep essential bills paid, I just can't afford frivolities anymore.
Darkwolf, if you didn't already you might want to include an ignore Th in that thing of yours.  (All Th endings indicate thumbnails)

[StarFighters76]

_09, Star Fighters 76, already a contributor here of course, but lately he wants you to get his maps off GameFAQs yourself so I may have a few you don't.  Of course you could just go to GameFAQS.

First off, sorry about the troubles you are going through.

However, I'm rather curious about this. Since I've hit 1,000 maps (on GameFAQs) 2 years ago, I've been in and out of the game mapping field, mostly because of being in school (and more recent, a really tragic accident I incurred myself). I honestly don't remember anything about this, but I won't object to it happening of course (VGMaps is a great mapping site). You probably go by another name that is more familiar to me

[JonLeung]

Gyaaah!  I shouldn't forget about this.  I've been so busy lately.  Anyone have suggestions for a good mass-downloader to make this easier?

[Grizzly]

If you just want to paste a list of links, Free Download Manager works fine for me. But for downloading from this specific .csv file and maintaining the folder structure, I used this selfmade piece of Java code instead:

Code: [Select]

import java.io.BufferedReader;
import java.io.File;
import java.io.FileOutputStream;
import java.io.FileReader;
import java.io.InputStream;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.TreeMap;
import java.util.TreeSet;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

public class DownloadVGMapperCSV {

public static void main(String[] args) throws Exception {
if (args.length < 2)
{
System.out.println("Run with java DownloadVGMapperCSV <csvpath> <savedirectory>");
return;
}
String filename = args[0];
String savedirectory = args[1];
if (savedirectory.equals("."))
{
savedirectory = System.getProperty("user.dir");

}
savedirectory.replace("\\","/");
char lastchar = savedirectory.charAt(savedirectory.length()-1);
if (lastchar != '/')
{
savedirectory = savedirectory + "/";
}

File load = new File(filename);
BufferedReader br = new BufferedReader(new FileReader(load));
String line = null;
ArrayList<ArrayList<String>> table = new ArrayList<ArrayList<String>>();
int part_i = 0;
Pattern patt = Pattern.compile("\".*?\"");
TreeSet<String> imagespages = new TreeSet<String>();
while ((line = br.readLine()) != null) {
if (line.length() > 1) {
ArrayList<String> parts = new ArrayList<String>(5);
Matcher match = patt.matcher(line);
while (match.find()) {
String ma = match.group();
parts.add(ma.substring(1, ma.length()-1));
}
if (parts.size() > 1) {
imagespages.add(parts.get(0));
imagespages.add(parts.get(1));
}
}
}
br.close();

String begin = "http://vgmapper.com/";
byte[] bytes = new byte[1000000];
int i = 1;
int size = imagespages.size();

for (String g : imagespages) {
System.out.println("Getting element "+(i++)+" of "+size);
String httpfilename = g.replace(begin,savedirectory);
String[] parts = httpfilename.split("/");
for (int k = 2; k < parts.length; ++k) {
StringBuilder sb = new StringBuilder();
for (int ind = 0; ind < k; ++ind) {
sb.append(parts[ind]);
sb.append('/');
}
File dir = new File(sb.toString());
if (!dir.exists()) {
dir.mkdir();
}
}
File file = new File(httpfilename);
System.out.println("File: "+file.toString());
if (file.exists()) {
//Picture already exists, must not be downloaded again.
continue;
}

URL imageurl = new URL(g);
HttpURLConnection imageconnection = (HttpURLConnection)imageurl.openConnection();
imageconnection.addRequestProperty("REFERER", begin);
try {
InputStream imageis = imageconnection.getInputStream();

FileOutputStream fos = new FileOutputStream(file);

int len = imageis.read(bytes);
while (len > 0) {
fos.write(bytes, 0, len);
len = imageis.read(bytes);
}
fos.flush();
fos.close();
imageconnection.disconnect();
} catch (java.io.FileNotFoundException fnfe) {
System.out.println("File not found: "+fnfe.getMessage());
}
}
}
}
(Precompiled version, ready to run)

But I am sure there are already professional alternatives that can download from CSV files? 

[Rew]

Trop, I know I'm late on this, but I'm tremendously sorry to hear about your parents. That's got to be awful.

[DarkWolf]

But I am sure there are already professional alternatives that can download from CSV files? 

The CSV was so Jon could filter out the things he already has based on the information Trop posted on user ids and such.  Once the list is filtered, the URL column could just be copied to a text file, and most mass downloaders I think would support that.  Of course if you want the downloaded files put in proper sub-folders and what-not, yeah you'll need something more advanced.

[JonLeung]

Just to let you know this is still going to live on on VGMaps.

This has been successfuly downloaded onto my XP computer thanks to DarkWolf and Grizzly's efforts (the precompiled downloader wouldn't work on my Vista computer or my brother's 7 laptop).  But my XP computer is the one with the chronic heating problems that no one can fix and I can't seem to simply swap the hard drive out.  Running approximately three or four applications at once is almost guaranteed to have it instantly shut down because of a "thermal event".

Now these are weak excuses at best, because I should still be able to transfer it all across the network (the USB slot heats up, and I suspect it of frying two USB flash drives already), even if it takes multiple transfer attempts, it's just that when I'm so frustrated with that computer, it's become more of a habit that I simply avoid using it as much as I can, so sometimes I forget, and when I do remember, I don't want to deal with it because it feels like such a monumental task, especially when I've got lots of non-VGMaps.com stuff going on in my life that I have to budget time for.  But I can see that VGMapper is important to you, Tropicon, and I shouldn't take your situation so lightly, so I apologize for that and that I haven't prioritized this as much as I should.

But if I deal with it a few games at a time (instead of seeing it as a huge bulk of over 5000 files), it won't be much more work added to the weekend updates, and so eventually the maps will all be on VGMaps.com.  I also have some time away from work coming up, which should help.

And writing this out here publicly certainly makes me more accountable...you guys can call me out if I don't regularly post VGMapper.com material.