Latest revision bugs

[bgbirdsey]

found something interesting. I added two integers to the end of the pip data structure and all of the sudden the number of particles that were used changed...

Does this mean that there is a bad usage of memset again, or some other buffer overrun?

[penguinflyer5234]

Buffer overrun most likely, but misusing memset can be a form of buffer overrun.

[bgbirdsey]

I looked at all of the memsets in the program and didn't see an error, but I found out that there is a way to do the notation that will probably save on errors (as long as gcc doesn't choke on it!)

if you have

Code: Select all

void some_function( some_type * ptr )
{
...
    memset(ptr, 0, sizeof(some_type));
// OR
    memset(ptr, 0, sizeof(*ptr));
...
}

and you use the second version you do not have to know the size of the type or even copy it properly. AFAIK this seems to be working well. The only pitfall is if you would say memset(ptr, 0, sizeof(ptr)) without dereferencing the pointer, in which case sizeof() just returns 4 (or 8 depending on the number of bits used to define your addresses).

----------------------

I am also in the middle of determining why the abyss 2 is sooooo slow in my debug version of the program. I tracked some odd particle behavior possibly to a bad application of memset, as explained in an earlier post.

At the moment, I am running the module without the scripting system, and certain particles are being generated way beyond all others-

• created 80955 - "/modules/abyss2.mod/objects/lostsoul.obj\part1.txt"
  created 6559 - "data/ripple.txt"
  created 837 - "data/splash.txt"
  created 649 - "objects/demon.obj\part3.txt"
  created 360 - objects/mephit.obj\part2.txt

zefz, are the lostsoul.obj things new?

I am going to rerun this tese with the scripting system on and see how many particles of what type are created. It gets complicated since the whole system is completely overloaded. I will report both the number requested, and the number actually created. These might be wildly different since a lot of particles are never spawning.


-------------------------

With the scripting turned on, the following partcles lead the pack:

• created 259157 - "/modules/abyss2.mod/objects/cursetrap.obj\part0.txt"
  created 80955 - "/modules/abyss2.mod/objects/lostsoul.obj\part1.txt"
  created 64802 - "/modules/abyss2.mod/objects/cage.obj\part1.txt"
  created 64789 - "/modules/abyss2.mod/objects/seal.obj\part1.txt"
  created 48592 - "/modules/abyss2.mod/objects/prisonseal.obj\part1.txt"
  created 32394 - "/modules/abyss2.mod/objects/daemonlord.obj\part4.txt"
  created 16197 - "/modules/abyss2.mod/objects/angel.obj\part5.txt"
  created 16197 - "/modules/abyss2.mod/objects/bishopseal.obj\part0.txt"
  created 16197 - "/modules/abyss2.mod/objects/thebetrayer.obj\part2.txt"

These numbers have been scaled so that the lostsoul.obj\part1.txt was the same number as in the precious test. 16197 particles corresponds to 1 particle per update.

I actually only got something like 5 frames for render and 5500 updates, so the program is getting lagged beyond all reasonable capacity.

[Zefz]

Hmmm. Lostsoul was added when Abyss 2 first came out. Nothing new. It's those faerie things flying around in the boss chamber.

Curse trap continuously spawns particles in a circle. Cage does the same. There are three cages or so. Seal also spawns a circle of particles every update. As does Prison seal, Deamonlord. Angel continuously spawns particles for her wings.

[bgbirdsey]

If nothing has changed, I need to look at the scripting system to see if something is taking too much time. I know that it was able to run in debug mode before, so maybe something stupid is happening in the code.

I do know that the debug version runs unnaturally slow when all of the free particles are used up. I tried to track that down, and it seems that there are no extra calculations in debug mode. It has been a mystery for a while.

But you do realize that that is 200k particles spawned in 300 seconds of operation?

[Zefz]

Hmm... That sounds a bit excessive yes. 667 particles spawned per second. Any suggestions on how to lower the number of particle calculations?

Maybe remove the particle glow around some of the objects? (They only serve as a visual effect anyways). Or maybe halve the spawning rate and double the life time maybe.

[bgbirdsey]

I'm not sure. The total number of spawned particles is around 1000 at any given time. If you download the newest svn you can see that it is displaying the fraction of time done for several portions of the graphics code, and the majority is due to the preparation for each loop and the drawing of the mesh.

It is most likely that we could benefit from trimming that code, maybe making the lighting calculation skip some updates or something similar.

However, the extreme level of slowness has come about rather recently. Given the amount of time that is being spent on various tasks, I think it is probably due to some scripting problem or a problem with preparing the frame for rendering.

[Zefz]

The cursed trap was spawning 8 particles every frame. I reduced this to 2 particles per frame and already that should reduce 12 particles per frame spawned (since there are 2 curse traps).

In the Abyss 2 I have an average framerate of 27-28 (opposed to 30-31 on other modules). The maximum expected framrate starts on -3 and increases to an acceptable number over time. It seems it's the first room that takes the most processing time. It's all the "water" and the huge amounts of mephits (each holding another mephit wing of course) that sinks down any performance on that level. As soon as I leave the room I notice an instant improvement

I noticed some particle bugs as well. Trails spawned incorrectly according to the main particle... Something about particles not being as bright as before. They look more like how TRANS particles looked before and TRANS particles now look like solid ones. (brazier flames are not transperant at all anymore for example). You were right about mind scrambler confusion rays not shooting properly either. It seems their main particle get stuck as soon as they shoot it.

[bgbirdsey]

That might be a problem with "air resistance" being applied to particles. It certainly seems like it. That should really be limited to things like arrows and coins (solid particles).

While I was messing around with the particle rendering, I changed the glBlendFunc() for LIGHT particles, and I made it so that both alpha and light affected the particles. It could be either one of these effects that is causing the difference. I will try to change it so that the particles are more noticeable.

Also, since the default number of particles is 1024, we need to be aware that we really only ave about, say, 700 particles for module use at any given time (leaving 68 particles for things like player weapon/spell effects, and 256 particles for "force" particles).

These numbers could be balanced better, since there is absolutely no way that we have ever used 256 "force" particles. Also it could be possible to make 3 levels of particle generation. For instance, particles that are not on camera could have a lower limit of maxparticles/2, and particles that are on camera could have a limit of maxparticles/8.

I can't think of anything that should cause problems with this, there are not even that many "scenery particles" like rocks and flowers in any of the modules.

[bgbirdsey]

I've done some analysis of the amount of time taken by various scripts.

When I am running the abyss2, I have a frame rate of about 1 per every 10 seconds or something. The actual scripting does not seem to be the culprit, as it takes only 20% the normal time for a single update.

In a way this is too bad because now I am no closer to decoding the problem

Anyway, I did organize the scripts by the "worst offenders"

Worst offenders by total time

• • 1.85s 11.92% "Plate"
  • 1.07s 6.93% "Module Control"
  • 0.79s 5.10% "Lord Bishop"
  • 0.53s 3.42% "Arrow Trap"
  • 0.43s 2.79% "Arrow Trap"
  • 0.40s 2.61% "Arrow Trap"
  • 0.36s 2.33% "Arrow Trap"
  • 0.30s 1.97% "Kursed Symbol"
  • 0.28s 1.79% "Kursed Symbol"
  the 1st column is the total time taken by the script, the 2nd column is the % if the total time for all scripts, and the 3ed is the name

Worst offender by average time

• • 100.00% "Lance"
    83.27% "Lance"
    79.98% "a Shock Bracelet"
    74.97% "Claw"
    74.69% "a book of Teleport"
    73.37% "Spork of Harmony"
    64.56% "a Darkshine Mace"
    61.97% "Module Control"
    56.27% "Mephit Wing"
    55.87% "a Book of Disenchant"
    55.85% "Deamon Wing"
    50.72% "Spiked Mace"
  the 1st column is the amount of average time relative to the worst

So, the question is whether any of these scripts can be optimized. It really isn't that important, given that the scripting system is coming in "under budget".

--------------------------------

I finally isolated it down to the move all particles loop

[Zefz]

That's pretty interesting. I will take a look at those scripts and see what I can do and which functions are causing the problem. (SetTargetToWhoeverIsInPassage?)

[Zefz]

I managed to optimize the Plate, Module Tool, Lord Bishop and the Arrow Trap. They all had the same problem. IfTimeOut was not always set every time IfTimeOut triggered, causing IfTimeOut to happen every frame. This was not a problem before when IfTimeOut only triggered once until a new SetTime was used.

I have no idea why or how any of the items such as the Lance, Bracelet, Claw or a book could be a offender. Maybe the feedback is wrong?

[bgbirdsey]

Looked at some more profiling of the rendering loop. The biggest offenders is the lighting calculations for the mesh and for the characters. Not too surprising since the calculations are quite a bit more complicated than before.

The biggest culprits are the chr_instance_update_lighting_base() function that calculates the basic character lighting based on whatever is lighting the mesh, and the light_fans() function which lights all of the mesh fans.

I was able to reduce the total graphics initialization time by about 33% by spacing out the character lighting calculations to once every 2 frames or so. That same stchnique can definitely be applied to the mesh lighting as well.

The only thing is that the constants for smoothing out the lighting (i.e. the 0.9 * old value + 0.1 * new value) need to be updated to take into account the longer amount of time between updates. I can do that in the graphics initialization code, so it is no big deal .

[Zefz]

I thought it was the lightning. The lostsoul character in the Abyss 2 spawn act as a light effect. There are about 15 or so lost soul objects that each spawn a lightning effect.

There are maybe other particles that spawn a light effect that shouldn't (like a fire aura or something silly like that)

[bgbirdsey]

Now you tell me! LOL

I think I may need to optimize that algorithm so that we do not do a bunch of useless calculations for every single tile.