Topic: Any idea why the Genesis "index.htm" file is a problematic file?

[JonLeung]

So I upload updated files to the site via FTP using FileZilla.

I have had a particular problem for a couple years now, where when I upload an updated "index.htm" file for the Genesis folder, it says it does... but the index file does not appear on the remote file listing in FileZilla, and the page disappears from the site, as if it was merely deleted.  So the way I've been updating it now is renaming it to something else, like "index.png", then uploading that with no problem, and then renaming it back to "index.htm".

I'm sick of having to do this extra step for only the Genesis (no other folder has that issue) so today I decided to delete the entire Genesis folder and start fresh, since my hypothesis was that the Genesis folder might have something weird in it, maybe ghost "index.htm" file fragments or something.  However, the problem persisted.  I deleted the folder again, made a new one with a different name (like "XGenX" and "uhhh") just in case it didn't like the name of the "Genesis" folder, with the intent to rename the folder back later if the upload worked, but no matter what, when I upload index.htm, it doesn't appear.  So, okay, I guess it wasn't the folder that had something within it preventing the upload, there's a problem with the file itself.

So I tried copying the file, and resaving the file as something else, just to make it "different", and yet it still has a problem.  I checked the file properties, and it's not hidden or anything unusual.  Even if there were some odd properties, why would being called ".htm" trigger the problem and yet ".png" or anything else does not?

What is the problem with Genesis/index.htm specifically that it can't be uploaded as it is, without renaming hijinks every time?

[dark_lord_zagato@yahoo.co]

What if you upload an empty file with that name? Like, make a new .txt file with nothing in it so it's 0 bytes and name it "index.htm"

If that won't upload then at least you can eliminate the contents of your file as a possible cause.

[JonLeung]

Great suggestion!

So I deleted the on-site "index.htm" and then uploaded a 0-byte "index.htm" as per your suggestion.  It accepted it with no issue.

I then reuploaded the proper Genesis/index.htm to overwrite it and there was no problem.

I was curious about this, as even overwriting didn't work before... so I deleted it and reuploaded it again to see if it could handle it without overwriting - and it worked!

Just in case this was a lucky FTP session, I closed FileZilla to start fresh and did it again and it uploads smoothly as one would expect - the problem appears to have been fixed!

Somehow the 0-byte file was easy for the server to accept and fixed whatever problem was there.  Which is weird since I really thought it was a problem with the file, when even new folders with other names had trouble.  Curious.  I still don't get what happened there... but at least it's not a problem anymore.

Thanks so much!     I figured someone would have an idea.  I should have asked this earlier...

[JonLeung]

Never mind... the problem is back.

Why?

So strange!  Guess I'm going back to the old workaround method... sigh.

Also, why does FileZilla have to shave off 12 bytes from .htm files when I upload them, but keep the filesizes the same for all other files?  Why does it have to do anything different with HTML files anyway?  Just... upload them.  That's like a delivery person going through my packages and taking away something probably insignificant.  It's like... that's not your job... just bring it where it's supposed to go.  By uploading the Genesis .htm as a .png and then renaming it to .htm, I notice it keeps all its bytes.  Maybe when I have some time I'll download an .htm file off my site and compare it to the original and see what it removes.  Maybe I should make a much smaller one so it's easier to tell.

EDIT: A quick Google search says something about different file systems like Linux vs. Windows and recognizing it as text or something.  I should go to bed now so I'll look more into it tomorrow, including trying to upload with the "binary" transfer instead of "ASCII" or "Auto".

[DarkWolf]

Is your web hosting using Linux? If so check your folder and file permissions. There exists scenarios in Linux where you can delete files from a folder because you have write access to that directory, but you can't overwrite a file in that directory because you don't have write permissions on that specific file. I would expect you'd see an error message with this sort of thing, but it's possible the server or maybe your client isn't properly handling the error or suppressing it for some reason.

Text vs binary mode can also create issues, but this usually has to do with white space or character encoding, that is, I would not expect the behavior you're describing. Doesn't hurt to try though.

[Grizzly]

The number 12 corresponds exactly to the number of newline characters in the Genesis index.htm. On Windows a new line is historically written with two bytes, \r\n, so the upload process most likely is converting the line endings to Unix style, removing 12 bytes.

Using binary transfer should help with this because it should exempt the file from any conversion (as does the Auto setting if it sees a file extension that normally corresponds to a binary file format; like .png).

The file replacement error also might have to do with the text processing and a hickup on certain special characters in the extended ASCII range. For example, Super Nintendo index.htm uses © (169), ® (174), ³ (179) and ÷ (247) while Genesis index.htm uses © (169), á (225), ï (239) and ü (252).
Here binary transfer should also help, or alternatively using the HTML replacement codes (e.g. á) and inserting them instead.
But binary transfer sounds more promising because as you said: the uploading process or server should not inspect the file at all but use it as is.

[JonLeung]

So I've tried binary transfer as well as replacing the special characters.

Interestingly, a single "©" that I hadn't previously replaced with "©" and the "ü" you pointed out to me were both with Lunar: Eternal Blue, and I believe that is the largest map set on the Genesis page, if not all of VGMaps, as Lüt did a whopping 791 maps - honoured recently as the Maps Of The Month for August 2021.  So it was quite the replacement!

However, the uploading issue remains.  How annoying.  And baffling.  Thanks for the info and suggestions though.

[JonLeung]

I think GoDaddy upgraded or changed something.  Four days ago, I was having trouble uploading the map for Last Battle on the Genesis - it's just one map and only a few other files like the index and the Atlas directory page and the news page.  It was just choking badly.  I eventually got it all through, but it took literally an hour for just those few files, which was ridiculous.

However, I've done a couple other uploads since that day, including a stealth upload for some updated/corrected Momodora: Reverie Under The Moonlight maps just now, and I noticed everything uploads faster - and with no errors!  Often with large HTML files like the NES, Super NES, and PC index.htms, FileZilla hangs after it gets to 100%, and then it tries to upload it from scratch again, so I have gotten to the habit of just closing out when it hits 100% but before it restarts on its own.  But now it doesn't do that, and doesn't shave off 12 bytes, either.  Just for fun, I uploaded the Genesis .htm again... and it works!  As it should!

I guess it's a little ridiculous to be joyous about something working the way it always should have.  But there we go.  Problem solved itself.  We may never truly know what the issue was, but I'll be okay if it never comes back.

[FlyingArmor]

Glad to hear this issue got resolved!

[JonLeung]

Well, then I noticed that this past Sunday (Hallowe'en night) when I was trying to upload the Maps Of The Month, it was literally just a handful of small files again.  I wonder if Sunday nights are just a bad time to upload, as it once again took a very long time.

Not long into trying the spotty uploading, I got onto the live chat with GoDaddy's customer service, who seemed to not understand "I can't connect via FTP".  I mean, it's GoDaddy - a web hosting service - meaning that I would think connection problems should be a common reason, at least among any reason, for anyone to contact them at all.  I don't know how I could be more clear, but half an hour in, they were like, "oh, can we transfer you to our tech team?" and then it took them another ten minutes to do it.  At the forty minute mark the I managed to upload the last of the files I needed with the extremely spotty connection, literally just as the tech person greeted me.  I still wanted to know why it was slow and the guy wasted another forty minutes (when I should have gone to bed) trying to find out what the issue was.  I know tech can take some time to figure out what the issue is, but he never sounded competent or reassuring the whole time.  I don't know if I was being a "Karen" about this but an hour and twenty minutes is a little ridiculous; I don't appreciate feeling like I got the runaround.

So, the next time the month-end is on a Sunday evening, I may have to upload earlier.

But hey, at least the Genesis/index.htm problem is gone... maybe?  Though if it were a trade, I wouldn't want the spotty uploading.  That's more frustrating because there's no workaround other than just trying again repeatedly or at another time.