Posted 14 August 2014 - 02:19 PM
Seperate map making rant, but loosely connected to mountains.
What i would REALLY love, is if the edges of maps weren't straight lines & WEREN'T framed by mountain ranges around it's entire limits of only ONE height. Looks so unnatural in every single way. Means there was no point in varying anything else as the effect is still lost.
Posted 24 August 2014 - 06:12 PM
The data is held in a 4x4 blocks of 128 bits with the first 64bits of each block being used to carry the alpha data. Alas its all in 4bit values which require some mangling to make them useable. Once that's done, it should then be a matter of scanning the data for any files that have all the alpha set to white. Interestingly, there's no need to scan the whole file. DDS files are comprised of a series of copies of the same image (known as mipmaps), each at a lower resolution than the last. Hence, scanning the first mipmap tells you everything you want to know.
Atm, i've got as far as locating the various mipmaps but, have yet to start testing the individual blocks. Just to test out my mipmap calcs, I coded a small dds image viewer using the glut library which, could be a useful additional option for e3d_conv.
Have now got as far as decoding the mipmaps for the DXT3 format which, as per the last post, is made up of blocks of 128bits or 14bytes. The first 7bytes of that deal with alpha data, with the first 2 bytes providing reference alpha values. By comparing these values, we can tell if alpha values are being used, ie where the first value (known as alpha_0) is less than the second (known as alpha_1). We then have to repeat this comparison for each block in the first mipmap, after which, we'll know if alpha is being used in the image. We don't have to bother looking at the subsequent mipmaps as they are just progressively smaller copies of the first.
As a preliminary, i'm testing the new code as part of the diagnostic report in e3d_conv and will be included in the next release. However, as the latter can't specify individual dds files (only the e3d through which the dds is referenced), i'm creating a separate stand-alone utility.
In researching texture compression, it seems possible that we could possibly convert some DXT3 to smaller and faster sizes by using the DXT1a format which, is essentially DXT1 with alpha. Unlike DXT3, which can handle multiple gradients of alpha, DXT1a only handles black or white. However, in many cases, that might be all that's necessary. In the meantime, the increase in client performance might be useful.
Whilst the client (as currently coded) can't handle DXT1a, it shouldn't be that difficult to implement. What is more challenging is that DXT formats are lossy, ie data is lost through the compression process when the file is created and, each time it gets edited or changed. Hence, there's a possibility that the conversion process may effect image quality. Not sure if it would be possible to programatically mangle the data without loss of image which, might be a way around this.
EDIT version 1.8 of e3d_conv now released. Source and 32bit binary download from https://sourceforge....er/version 1.8/
Posted 20 April 2016 - 03:43 PM
I remember making this thread wayyyy back ages ago. It started off being about a new client and then kind of ended up focusing on tools to convert EL's e3d graphics format. Not much to say on the latter other than the separate tools that were used to convert from e3d to obj and vice versa were combined into a single tool called ul_conv, which you can find at http://unofflandz.co...lay.php?fid=20or https://github.com/U...-landz/releases
Learner - if you'd like to create a specific OL version you're very welcome to use the codebase, or even kick me and say 'Munt - will you do them for me'.
Anyhowz, the orginal subject of the thread was about developing a new client, so I thought I might as well reuse the thread to update peepz on where i'm at, rather than creating a brand new thread. So yeah, it's got lots of developer related stoofz which some peeps may find more interesting than others. I guess the point of making the post, apart from the joy of making long self-indulgent monologues, is to involve the wider community and let them know where i'm heading and why.
So some background. The official OL project intends to build the new client from an engine called Unity. Nothing much wrong with that except it's a closed source engine which has to be scripted in object orientated C#. Whilst i'm not entirely unfamiliar with C# or the whole object orientation paradigm, I don't want to have to bee bothered with that kind of thing at the same time as grappling with the monster that is working out how a 3d engine actually works. So, I opted for an engine (Irrlicht) which is coded in shitty C++ (which I hate), but which is sufficiently C-like for a mere mortal to understand. Actually, i'd have gone with Horde3d which is significantly C-Like and properly fast or; Raydium which properly C99 (yay), alas both suffer from lack of documentation needed for a complete nublet to get a clue. Irrlicht, on the other hand, has masses of documentation and even several books.
So I started on my experiments with Irrlicht which, unsurprisingly involved trying to import an OL map and render it somewhat like it looks in the current OL client. That ofc is easier said than done. The elm format used to store OL maps is a EL proprietary format that only the OL/EL client's understand. Extracting binary data from an elm file isn't massively hard, but wrangling it into something that can be understood by another 3d engine turned ut to be more challenging than I imagined. Essentially, what you have in an elm file is three structures, ie:
a ) a tile map
b ) a height map
c ) a 2d object list
d ) a 3d object list
The tile map is the easiest of all to deal with. OL maps consist of a grid of tiles. in the case of Isla Prima, the grid is 32x32. The tile map consists of a linear array of values corresponding to the texture for each tile. You can find these textures in the 3dobjects directory of your OL client installation in the dds files named tile1.dds to tile242.dds. As the data type used to store each entry in the tile map is an 8bit unsigned integer, the maximum possible tile values are 255, so it would seem that we're already pretty much at the limit of what the current client can support.
Rendering the tile map is fairly straight-forward, after which you get something that looks (in the case of Isla Prima) something like this :
Yup, simple enough. However, next is the 3d object list and it's here where things start to get gnarly. For starters, the structure is a mix of text and a variety of binary values, some floating point, some integer, some 32bit with a few 8bit thrown in for good measure. However, C coders laugh in the face of this kind of challenge and, with the magic of structs, we are able to wrangle this into a series of values specifying the coordinates of each object, it's rotation and the name of the mesh file (e3d) which draws the object. Yup, look in the 3dobjects directory of your OL client installation and you'll find a whole load of files with the e3d sufffice, each of which specify how a particular 3d object should be rendered.
Warning LIES ALERT: actually no. E3d is another of those shitty proprietary EL formats that only the EL/OL client understands. Try asking your favourite 3d modelling app to read an e3d and it will produce a message from Barclays Sperm Bank reminding you why your account is well and truly overdrawn. Hence, our guilty secret is that all the e3d's have been converted to wavefront obj. Worse still, all the accompanying texture files are converted to png as the compression formats used in the dds files are patented, not by radu, but by the kind of peeps who can and do collect revenues.
So once we've wrangled our object coordinates, first thing we discover is that the values look too small. The second thing we discover is that the y and z axis values are misplaced. Hence, we swap the y and z values and then multiply then double the x and z values. Next we deduct 25 from x and 26 from z gets us (near as damnit) to the right place. Alas, when we render, everything looks too small, hence, we scale up the x and z axis by 17. Also, because everything looks kind of flat, we scale up the y axis by 13. So now our render looks like...
Ok, all that was comparatively easy. Now for the uber gnarly 2d object list. Just like the last list, it's a combination of text specifying a 2do file and floating point values specifying position and rotation values. However, first you have read the 2do file which is text based and looks something like this
So what the above specifies is a particular part of a kind of image file known as a texture atlas, ie a single image containing lots of different textures. Here's an example...
As can be seen, there's four different dirt patches in the image, one of which is referenced through the parameters given in the 2do file. Extracting the image is easy enough, simply use u_start and v_start to determine where the texture starts and then u_end-u_start and v_end-v_start to determine the height and width. In fact I wrote a little application that does precisely that...
N.B The little sliders are so that you can edit existing or create your 2do's.
So far so good, but rendering your 2do's in the right place is problematic. Once again, you have to swap the y and z axis, multiply by 2 and add on some for the Queen, otherwise you end up with this ....
However, even once you've corrected the positioning, there's still weird artefacts with the rotation. A good example is grass borders. whilst some are properly orientated, others look like ...
Clearly there's something going on that I don't yet understand, particularly when you look at the gang plank in this shot...
and there's a trio of weird doors....
Yup, so clearly things are still a 'work in progress', but still progress is being made.
Posted 20 April 2016 - 04:06 PM
Posted 20 April 2016 - 04:26 PM
Posted 21 April 2016 - 04:57 PM
I remember the pain involved in working out how to do a windows build of the current client. I shan't miss that at all. Anything that makes Xplatform builds easier has to be a good thing.
And you'll be pleased to know that i've started experimenting with the dubious delights of C#/Mono under Unity.
Posted 24 April 2016 - 08:42 AM
Firstly, I managed to work out the reason for the 'trio of weird doors'. Seems that anything on the 3d object list which has a blended value of 20 is in fact redundant data that should be ignored. Hence, excluding this redundant data not only removes the weird doors, but a lot of other artifacts. Actually, thanks for this needs to go to BlackKat, as it was only as a result of reviewing the code for the elm2pov converter mentioned in his post that I got an insight into what was going on.
I also managed to sort out the misplaced 2d objects so as the various grass and dirt borders now all line up neatly, eg:
What i've done is to deduct 180 degrees from the z axis rotation where where that rotation is between 45 and 135 degrees. This seems to cure around 95% of all the previous artifacts, although a few still remain, eg :
The end of this path is a mess
There's a strange piece of grass jutting out into the garden
Not quite sure what the reason is for these few strange artifacts. I suspect that it maybe has to do with some subtle difference between the way OL/EL and Irrlicht handle mesh rotations. Anyhow, some more experimentation is necessary.
So a new issue that's becoming apparent are the rather distinct borders to the 2d objects. A possible cure for this may be to increase the alpha in the object so that the object border becomes slightly transparent.
I've also started messing around again with the water plane. I'd post some of the results but I think i'm up to my maximum images on this post. In any case, the results are appallingly shit at this stage. Part of the shittiness arises because creating waves on the ocean also results in similar waves on the ponds, and which looks ridiculous. I could just have one boring water texture for both, but it would be far nicer to have waves on the ocean and ripples n the pond.
So now i'm getting towards (what is hopefully) the end of the map rendering, next step will be to try and add some map editing functionality.
Posted 24 April 2016 - 10:11 AM
Also, I would think the outlines would be brought about by increases in height or tilt angle, but I could be wrong.
Posted 28 April 2016 - 10:37 AM
Posted 28 April 2016 - 06:56 PM
Tried height adjustments and slightly changing rotations but it didn't help. Eventually, I did solve the issue.
The problem was in fact two things. Firstly the level of alpha and secondly the textures sizes were not 'power of two'. The first part of the problem i've solved by slightly increasing the alpha content to make the texture partially transparent. This makes the texture borders less well defined and therefore appear seamless (see below)
However, when you get really up close, you can still see vague artifacts that arise from the texture being slightly stretched. That's because some of the 2do parameters are not quite 'power of two', ie divisible exactly by 2. This results in the GPU slightly stretching the texture. I'm currently finding the best way to fix that. Modern GPU's also work best if the width/height of a texture is in the ratio 2:1 or 1:1, hence i've also to see if that can be achieved.
Posted 02 May 2016 - 03:47 PM
So i've completed the code that detects what mesh the cursor is over (it can even detect the specific triangle in the mesh). Atm, the detected mesh is rendered in wireframe (as per Stardarks original Unity code). Only problem is that this isn't a good way to deal with tile editing as the required functionality requires the user to see changes in texture. I've therefore got to find a better way to highlight the selected tile, either by outlining or, by increasing specular luminosity.
At the same time, i've also got to start work on creating the basic gui allowing the user to select from a range of different textures and which will require the screen to be split into separate sections (plus a menu bar at the top).
Finally, i've still got to deal with the rotational issues that are causing the few remaining visual artifacts (like the reversed gangplank and weird grass strips).
Posted 03 May 2016 - 12:02 AM
Was your original solution for the backwards objects to add 180 degrees to get them to flip? It seems like that is what's wrong with things that were already rotated: now those objects are backwards :s
Posted 05 May 2016 - 11:32 AM
This isn't an excuse, actually i'm totally appalled and embarrassed. The problem arises simply because I am a ridiculous arse who is failing to understand something at a basic level. However, before I wallow in self-pity, i'm going to shatter the existing OL client code into a million tiny shards until I get an answer.
This is the way of the troll. We smash our stupid heads against the brick wall until it simply breaks.
Posted 05 May 2016 - 01:23 PM
Posted 05 May 2016 - 01:53 PM
Posted 05 May 2016 - 02:48 PM
Posted 05 May 2016 - 03:02 PM
Posted 06 May 2016 - 12:00 AM
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users