Topic: Crusader: No Remorse

[Weatherproof]

Update!!!
So the past month at work I've been learning programming in VB .NET and C# and doing custom applications for stuff that my boss needs.  And one of them involved analyzing images, breaking them apart, and stitching them back together.  So after some minor adjustments, I've been reworking the tool for this Crusader project.  But I think it will really really benefit anybody stitching images together on this site.  It uses the Accord.NET libraries and is using some of the code taken from this site:
http://www.codeproject.com/KB/recipes/automatic_panoramas.aspx

It's definitely a cool thing to check out.  I'm trying right now to be able to dump a directory of images into the program and have it stitch a mosaic together.
Here's a screenshot of it working currently:
http://img545.imageshack.us/img545/4610/autostichv1.png
So right now, it takes two images, stitches them together by corner detection and correlation, then blends them (you can see the blank alpha sections where neither picture covers.  The main problem will still be that goddamn HUD.

[Maxim]

If the HUD is in the same place on every image then you can make the code ignore those specific pixels (in my autostitcher I call it "masking"). Of course, locations where there is no overlap of the HUD area will be left blank.

Is the engine based on fixed-size tiles (on a grid) or are the "pieces" of varying sizes/in non-grid positions? I have a tool that can fill in gaps in tile+grid images.

Is there anywhere in the game where the HUD is disabled? Cut-scenes, for example?

How does your stitcher scale to large numbers of images? I'd be worried it'd get ridiculously slow as the overall image gets large. Does .net/Graphics/etc fall over when images get very big?

[Weatherproof]

After taking a day off to spend on this, I've still gotten nothing to work.  The images are warped and distorted as soon as more than 3 of them get stitched.  I'm also having great trouble automating the process.  So in the meantime i'll have to find another corner detection/edge detection algorithm that is better suited for this set of images.

To answer your questions Maxim:
The HUD is in the same place every image, and that masking idea is great! I'll implement that eventually

The engine is tile based, I think...
And the HUD is never disabled (Cut-scenes are FMV)
And until I get this to work I won't be able to tell how it scales, but I'll report here first thing.

[Maxim]

The problem may well be that most photo stitching algorithms expect to align things on fractional pixels (with some warping too), whereas screenshot maps are always perfectly aligned and not stretched/rotated/etc. Maybe you can apply some rounding to the match locations and/or disable warping (maybe discard the homography matrix stage).

[DarkWolf]

After my failed attempt at writing a pixel-perfect automated stitching program, I thought a GUI assisted one might be better.  It would allow you to drag and drop an image near the area where it should go and it would attempt to match in that general area.  The end result would be similar to a grid snap, without a grid.  Would improve the speed issue since you could take the first match and not worry about the rest.  Still, I do web development for a living and not standalone applications, so my skill with doing applications that require that kind of re-painting isn't too great.  Would be nice to see someone else try it though.

[Maxim]

You need to start with decent large-image support, a lot of frameworks fall down with 10000x10000 images (for example). My autostitcher would be a decent base for it - it's designed to start at a guessed location and then look at increasing distances until it finds a good match. Feature recognition might be a faster technique than brute-force pixel matching.

[Weatherproof]

So i've done some more work on it.  I took out the homography estimation stuff because there's no point in using that on a flat image, it actually warps and worsens the result.  The homography is great for panoramic images but is useless for flat images such as video game maps.  The program now detects corners (Harris detection with Accord.NET still), correlates them, and instead of homography and ransac analysis to fit the inliers, it just subtracts the correlation points, then finds the majority of where the corners want to go and ignores the rest.

The result is a non-warped image that aligns perfectly, computes faster, and will work with any size image.  The next thing to do is implement a batch process so i can open a directory of images, and optimize stuff here and there.

The Harris detection is kind of expensive for large pictures so either i reduce the area searched, or move to a more simple detection algorithm (which is possible because there's no noise with these images), this would require me to actually write the algorithm though.

[Maxim]

For feature detection you could run a 2D edge-detection convolution and then maybe erode and threshold it. You could then build up the overall "feature bitmap" as you go along by merging in the matched images. You still have the problem of having to search globally (unless you assume some locality of sequential shots, as I do) and the fact that low-contrast areas will have few or no "features" by this algorithm.

[Weatherproof]

Another update:
The program will now load a group of images, they can be any size and there can be any number of them.
The program will use a bunch of logic to determine the canvas size, then it will place the base picture, then overlay the second picture by the majority offset.
It then saves the new image as the new base repeats for all the pictures.  Because the base picture gets successively bigger each iteration, the program takes longer and longer for more pictures.
However, i did some profiling and found that the slowness is 99% the harris corner detection and the correlation (from the Accord.NET library).
So I'm going to code a custom corner detection (actually copy the Moravec or FAST algorithm).
The images also need to be loaded so that each new image will overlap at least some portion of the previous image (ie: you stitch some images together, then the next image is in a different part of the map), so I'll just have to do some really crazy logic to sort the pictures or something first.

Hopefully, this program will greatly aid anybody on this site stitching far away images together that need to be pixel perfect.