SoF Development Tracking

[trevius]

I figured I should start a thread to help keep track of the development of SoF. This can be used as a reference for what needs work, and who is working on what. I will try to keep this post updated as people update the thread with any progress or notes. Please try to keep posts in this thread for developer work only. All other SoF development discussion should be posted here, or a new thread created for it. For reporting Bug in Secrets of Faydwer that are not already listed in this post, please post them in the Secrets of Faydwer - Bug Reports thread.


Last Update - March 5th 2010

Top Priority Work:
1. AAs - All AAs in SoF are now fully functional. A temporary work-around is in place for the Improved/Advanced upgraded versions of AAs. For now, we just see double of the base AA for the upgraded version, but they work and can be purchased. This is because SOE consolidated those into single AAs. We just need to figure out how Live handles it to deal with training them higher instead. More notes in the post below.

Medium Priority Work:

Low Priority Work:
1. Adventure Leaderboard - The stats of the Adventure Leaderboard don't work properly yet. I think the correct opcode is set, but it just needs an encode and structure adjustment.
2. Levitate - Levitate seems to cause PCs and NPCs to hop if it was cast prior to you zoning in. Levitate is now supposed to go in the spawn struct as "2" in the flymode field instead of from a separate appearance packet like Titanium uses.
3. Animation - Animations such as sitting, crouching, death do not show up if someone zones in after the animation was already used. This is because of the new StandStance field in the spawn struct for SoF being hard set to 0x64, which is standing. We need to set it to put the actual animation being used in that field instead of forcing it to standing.
4. Inspect Notes - Inspect works, but the player notes in the inspect window do not work for SoF clients yet. I think this may need to be set in the player profile now or something.

Database and Source Code Changes that will be needed at some point:
1. AA Table - The alt_vars table may need further adjustments to allow it to be fully functional for both clients and their differences.

Other Issues:
1. The SoF client seems to have a few crash issues. No real details at this time on what causes crashes, but it is still a bit less stable than the Titanium client is.

New Systems to SoF that did not exist in Titanium:
1. Blocked Buffs - This should be pretty straight forward to get going at some point.
2. Auras - These appear to mostly work like Songs, but also have an aura NPC that can zone with characters.
3. Fellowships - The NPC class that gives fellowships is Class 69. The window opens up for the client already, but nothing is actually coded for handling them yet.
4. Pirate Combat - No Information
5. New Power Source Equipment Slot - Power Source Slot added and stats from it are functional, but it does not yet work the same way it does on Live.
6. Guild Banners - Banners are spawned as NPCs and are a global race. Custom Guild Banner info should be in the spawn structure somewhere.
7. Destructible Objects - No Information
8. Player Traps - These are spawned as new global NPC races. Most likely, they are just quest scripts in the default.pl to control the trap actions.
9. Spheres of Influence - No Information
10. New Skills - Unsure how many new skills there are in SoF, but live has Triple Attack as a new skill (skill 76). Not sure how to add these to work with both SoF and Titanium.
11. Respawn Window - Have started work to getting this added. It isn't functional yet, but getting there.
12. New Item Stats - All possible item stats now show up on items when a value is set in the database. The new SoF stats still need to be coded into the server to actually be used. KLS Added Backstab Damage, but the others still need to be coded.

Recently Resolved Issues:
1. Consumption - Food and Drink consumption is now working and is now consumed at the correct rate.
2. Trading (to NPC or PC) - This is now working.
3. Objects - Interacting with objects (tradeskill, aug pools, etc) is now working.
4. Interrupts - Spell interrupt is now functional.
5. Groups - Grouping should now be fully functional.
6. Banking - Banking is now fully functional including the 8 new bank slots, shared bank and slots within bags in any of those slots. This also resolved a fairly major crash related to banking.
7. Clickies - Right Click item effects now seem to be fully functional, including observing recast delays when they apply as well as using item charges when they apply.
8. Drakkin Corpses - Drakkin and Froglok corpses are now showing up properly after zonedumps (thanks Derision!)
9. Tasks - The Task system is now fully functional (thanks Derision!)
10. Pet Buff Window - Pet buff window is now fully functional (thanks Derision!)
11. Animations - Animations are now fully functional (thanks Derision!)
12. Tracking - Tracking is now fully functional (thanks Derision!)
13. Skill Points - Skill training points now report properly (thanks Derision!)
14. /who - The /who command is now fully functional (thanks Derision!)
15. Item Links - Item Links are now fully functional, including Item Links from command outputs.
16. Item Augments - Item Augments are now fully functional in every way (HUGE thanks to Derision!)
17. Money Trading - Traded money now updates without having to zone (thanks Derision)
18. Bazaar - Bazaar trading is now fully functional (thanks Derision!)
19. Loot Slots - Looting corpses with multiple items will now always loot the correct item
20. Tribute - The Tribute system for personal tributes is now fully functional (thanks Derision!)
21. Rest System - After 30 seconds of being out of combat, extra regen kicks in to regenerate characters mana/hp/end at faster speeds. (completely new system - thanks Derision and drakelord!)
22. Books - Books are now functional (thanks Derision!)
23. Attunable Items - Attunable items now function properly in SoF (thanks realityincarnate!)
24. Spell Interrupts - Spell Interrupts and Fizzles no longer cause abnormally long refresh times on the spells being cast.
25. Stun - Melee stun/bash no longer causes unintended spinning
26. Spawn Structure - All of the currently desired fields should now be identified (thanks for the help, Shendare!)
27. Drakkin Specific Features - All Drakkin Related fields (Heritage, Tattoo, and Details) are now fully functional for NPCs and PCs.
28. Potion Belt - Potion Belt seems to function perfectly fine now.
29. /bug - /bug is now functional (thanks KLS!). Verified that it works in SoF with no modifications required.
30. /petition - The /petition command in SoF has changed to now open a browser to the SOE support site. Apparently /guidehelp has replaced /petition and functions in exactly the same way.
31. Chat Channels - Chat Channels and Mail are now fully functional in SoF with the addition of the new UCS (Thanks Derision!!!)
32. Item Stats - The item table now contains all of the fields for SoF including Heroic Resists which aren't used on Live but are built into the client. All stats now show up on items if a value is set, but no code is in yet for the server to actually use the new stats.
33. Tradeskill Combines - Tradeskill combines now work in tradeskill containers within your inventory like medicine bag or warrior epic 1.0 sword combing/split.
34. Recast Count-Down - The recast count-down on clicky items with recast delays no longer leaves items greyed out all of the time. This feature is available in Titanium as well, but doesn't work for Titanium yet either, so I am marking this off of the SoF specific issues list. Though, it would be really nice to see this get added at some point.
35. Raids - Raids are now functional in SoF (Thanks KLS!)
36. Death Disconnects - Dying in the same zone you are bound in no longer causes you to be disconnected (Thanks KLS!)
37. Guild Management Tool - The guild management now shows members and member notes and should be as functional as the one in the Titanium client (Thanks Derision!)
38. Stamina/Endurance - Live now sends endurance update packets and SoF probably does as well, so it wouldn't hurt to add an update packet for SoF. For now, endurance seems ok in SoF though.
39. Azone2 - Now functional for the EQGv4 zone files, so every SoF zone is now usable, though watermaps still do not work for EQGv3 or EQGv4 zone files yet (Thanks Derision!!!)
40. Max Level 75 - A work-around for level 76+ was added for SoF to correct HP/Mana/End by sending a tribute packet that sends what the stats should be (Thanks KLS/Secrets!)
41. Inspect - Inspecting SoF clients is now functional. The only functionality it is still lacking is the ability to set a note in SoF client inspect windows.
42. Clicky Checks - Items with click effects now have the proper checks for all clients to verify that they are allowed to be used.

Attaining Secrets of Faydwer:
Everquest: Secrets of Faydwer can currently be purchased from www.newegg.com for $19.99 with free shipping.
http://www.newegg.com/Product/Produc...20of%20faydwer
There are also various other sites that have SoF for sale at lower prices than Newegg.

This post will be updated again regularly with current development status.

[KLS]

Azone2 probably wont load some of the newer zones as they moved to version 4 eqg at some point and azone2 can only do <=3.

I'm gonna take a stab at item packets soonish once I can get client setup.

I'm at a loss on how to handle drakkin, since titanium and 6.2 wont be able to display them at all. I guess we could just have them pretend to be human for <SoF

[trevius]

Thanks, KLS. The items are one of the last major remaining items to get working before players may be able to start testing it and reporting bugs. I think I should hopefully have the AA and /who issues worked out really soon. Once items are done, I think combat just needs to get worked out and basic play should be functioning. For all other stuff, it will mostly just be opcode finding I think, which shouldn't be too bad at all.

I am actually surprised how close to being ready this is already. As long as there are no more major bumps, I don't think it is far off at all. Maybe Azone can be updated sometime in the future to work with the new EQG files. Other than that, I think a couple minor code adjustments to handle the new player race and maybe a couple other things, and then a couple minor database adjustments to handle some of the new stuff as well. Then it should be fully functional. At least as well as Titanium is for us now.

And yeah, for Titanium players, they will just have to see Drakkin as a human model or something. The Drakkin race is model 522, which is well beyond what Titanium can use. By default, they would see the frozen arms and legs out human model. But, we can probably force them to be any race in the Titanium/6.2 client cpp files in the spawn encodes. I think it would work doing a race check when the spawn packet is sent, and then just convert any race that is > 473 to be race 1 instead. Really, it isn't a huge issue, because they won't be able to see the new weapon models and stuff that the SoF users will be able to use anyway.

Eventually, I imagine most people will just move onto using SoF instead of Titanium. Especially once it is just as functional as Titanium currently is.

I must say that after going through each of the 81 new zones in SoF that have been added since Titanium, I am really amazed. SoE has done an excellent job with their zone design. I didn't care much for the Buried Sea zones, but other than those, all of the other expansions really wow'd me. And with 103 new races to use, I think this will open up alot of doors for custom servers. I haven't even got to seeing the new weapon models yet, but I imagine they are great as well.

After SoF is done, I will most likely look into seeing if I can possibly fix EQExtractor and EQBuilder2 to work with EQLive again. Then, we can use them to populate many of the new zones. I think that combined with SoF will help keep the emulator thriving for a long time to come. And since it is pretty likely that it will be the final client version for the emulator, it should make it worthwhile to work towards a 100% functional emulator

[Derision]

Now that I'm SoF enabled, I'll pick up Character Creation if no-one else is working on it.

[trevius]

Thanks Derision! I am glad to start seeing some dev interest in this project. It has been a pretty huge one so far lol. The amount of hours I have put in over the past couple of months is pretty insane. I was hoping once I got it to a somewhat playable point that others might start realizing that it wasn't just a pipe dream anymore and that it was going to actually get done!

If you need any help or suggestions for character creation let me know. I can easily get ShowEQ logs of the character creation process from EQLive to share with you. Those will be much easier to read than packet sniffs from WireShark or something. That is assuming that EQLive and SoF handle it the same way, but I am pretty sure it would be the same, or at least much closer than how Titanium handles it.

I have been focusing on AAs mostly over the past couple nights. I am pretty sure that the structure is just wrong. I was basing it off of the EQLive AA structure, but I bet SoF was slightly different. The good thing is that there is minimal difference between EQLive and Titanium, so it shouldn't be hard to figure out what to change/remove from the current struct I have that is based off of EQLive.

I have also filled in a few more Opcodes, but haven't updated the SVN with them yet. Probably 70% of the Opcodes we need to fill them all out will come directly from the client in a tail log (as an OP_Unknown), so those will be very easy to get. For the rest of them, they may be a bit trickier, but those can be handled when the time comes.

[Derision]

I got Character Creation working, mostly.

There is a new OpCode the client sends when you click on 'Create New Character'. I got a trace from live and the server responds with a 22KB packet, with what I assume must have valid starting stats, starting zones, deities etc, for each race/class.

I didn't bother trying to interpret what is in the packet just yet, I just have a hardcoded response with the same 22KB of data that live sends which is good enough to get into the character creation screen on SoF.

I can currently only select races from the original game ... the Expansions data must have moved within whatever packet it gets sent in, also the Enter Tutorial button seemed to be missing. Both will hopefully be easy to sort out.

The way the initial start zone is requested also seems to have changed, so everyone starts in South Qeynos for now.

[trevius]

Detailed Notes on Subjects Above

AAs - Currently, there is a fair amount of work that needs to be done to get AAs fully functional and up-to-date in SoF. Most of them now work thanks to a structure issue that Derision found and corrected. In order to handle AAs properly for SoF, we need to create and use new tables just for SoF AAs. Right now, AAs load into the AA window and can be purchased without having to zone for them to update.

The total spent AA count in the AA window is temporarily being forced to show 500. This is because I do not know how to get an accurate count of the total spent AA points yet. This is a fairly minor issue, but worth mentioning. Derision found an issue with the encode of the RespondAA packet, which is what sends the spent AA points total. So far, I now have it able to show the correct number of AAs purchased, but not the total of AA Points spent to make those purchases.

The main priority on AAs right now is to get the table issues worked out. I have already collected every AA that a rogue gets on Live and have put them into a table format. I was able to get them working in game as well, but need to do some clean up on them before they are ready for other admins to use. The table I have so far for this is posted later in this thread. At some point, we will need to collect the AA info for each class and put them into the table as well. It seems that the AA data for Titanium still mostly exists in SoF (since AAs show up), but SoF seems to get descriptions and names and such from different places than Titanium does. Titanium uses the eqstr_en.txt file to get most of it's descriptions, but SoF uses the dbstr_us.txt file for them.

Something worth mentioning is that it seems like getting AAs set in the database might be easier for SoF. The dbstr_us.txt file is setup differently than the files that Titanium uses for AA names, descriptions, and maybe other info. Basically, it seems to use a list format for each ID. For example; Innate Strength is skill_id 2, and in SoF, the name and description in the table can both be set to 2. This is because the client knows that the name is always the 1st line in the list and description is always the 4th. Here is an example from the file to explain what I mean a bit better:

2^1^Innate Strength
2^4^This ability raises your base Strength by 2 points for each ability level.

So, skill_id, title_sid, and desc_sid all get set to 2. This just means a little less work for getting the AAs added to the alt_adv table. Nothing major, but seems like a better way to do it.

I think I have the alt_var table I have so far is looking pretty decent, but still need to look into the aa_effect and aa_action tables. Once all of the AAs are collected for all classes from Live, I am sure we will have to do some clean up to make sure everything matches up and is set properly.

Once the new tables are ready, I will probably need help from someone more experienced to get the SoF encode to use the new AA tables instead of the old ones. We will also need to have it send a couple new fields that didn't exist in Titanium. I figure I will deal with that once the tables are more ready, though it won't be long now.

It seems that with the change to how the AA window works since Titanium, it means that the way that the AA tables are sent works a bit differently. As far as I can tell, none of the actual AA IDs listed in AA.h have changed. But, since many AAs were combined/consolidated to reduce overhead, the way they are handled had to be changed as well. The most noticeable change is for stuff like the Innate stats, and other AAs that used to have direct upgrade line AAs for them, but are now combined into a single AA. The issue is that with the IDs remaining the same, it doesn't allow for them to be combined in the same manor that Titanium handles them individually. To use Innate Strength as an example, the ID for it is set to 2, and the next AA is Innate Stamina, which has an ID of 7. This means that there is only room for 5 levels of that particular AA. In Titanium, the max_level for that AA is 5, so that works fine. In EQLive (and probably SoF), the max is now 15 for Innate Strength. It seems that each time you train a point, it just moves up 1 in the AA effects table. And, to compensate for this restriction, it looks like EQLive just sends that same AA table 2 or 3 times so that the effects stack. So, if you had 1-5 points trained, it would send that AA 1 time, if you had 6-10 trained, it would send it 2 times, and if you had 11-15, it would send that same AA 3 times. I am not 100% sure that this is how it works, but it definitely sends multiples of some of these, and it only seems to do it when you have points trained in that stat. This will definitely be something we will have to understand fully before the new SoF AA tables can be completely finalized. I still have to figure out exactly how each table is supposed to be formatted to handle this new way of doing it. I think most things are still the same, but changes will definitely have to be made here and there.

I just noticed that the unknown0092 field in the SendAA structure seems to actually be a secondary category field. The first category field is currently labeled as "expansion", because it is the number that correlates to the expansion that the AA originated from. The unknown0092 field seems to override the expansion field if it is set to anything other than -1. I am going to do some testing with this in SoF and if it works, I will label the field "special_category". So far, I have identified that it will show the following if the field is set to any of these for SoF:

unknown0092
2 - Progression
6 - Tradeskill
7 - Expendable
8 - Racial Innate

As with the other new fields, this one will have to be added to the code for SoF AAs to have it sent properly. I already have them listed in the new tables I am working on.

That leaves the unknown0096 field as one of the only new unknown fields that hasn't been identified yet. The only AA I show as setting that field to anything other than 0 is Harmonic Dissonance. That is an AA on live that is free to purchase and will port the player to a zone. It is currently set to 16777216, but I am not sure what that number is supposed to relate to yet. This field probably isn't very important to us.

Recast Count-Down
Clicky items that have a recast delay on them all seem to show up as grayed out all of the time. This may be due to an issue with the serialization portion of the struct, or possibly something in the click effect portion that is not getting set properly. Interestingly, it seems that after my recent changes to the manachange encode, my click type 2 items now count down as they should. This could possibly mean that the manachange packet is what updates items to start the countdown after the click happens. If that is true, then we should just need to figure out the correct values to use in the unknown fields to correct this. I have it hard set to -1 right now, so I don't know why it would effect recast type 2 only. Maybe it is just coincidence that they work now or I may not have noticed that they started working after some other recent change. Either way, if we can figure out why type 2 works, then we should be able to figure out how to get the others all working. And if we can get it working in SoF, it could possibly be fixed in Titanium as well, because I know that feature was in Titanium. This system isn't crucial to get working, but it would be handy to have

[Savnok]

J&R Music World has copies (new) of SoF for $10.92 shipped. That is the best I could find currently and even beating the Ebay.
J&R is a fine fine company and you will get yor DVD. They have been around forver.
No I do not work for them ..life becoming a 24/7 Geico commercal for you? LOL, I know.

$10.92 shipped new DVD SoF copies. I am hearing this DVD is becomming extinct soon. Ouch

[provocating]

When I bought mine from NewEgg for $5 shipped, I almost ordered 3 or 4.

[Rogean]

Just deleted a message, keep in mind:

Warez Is NOT Allowed
-This includes the transfer of any copyrighted material, including EQ Files and backpatches. Requesting files, even in a PM, will result in a ban.

[Greyhelm]

It was me. My apologies. Guess I should read the rules. Thanks.

[thepoetwarrior]

I got my copy on ebay brand new sealed for $5

[Derision]

Medium Priority Work:
1. Stamina/Endurance - Stamina/Endurance does not regenerate properly and seems to drop when you zone. I believe this is because Live now sends endurance update packets and SoF probably does as well, but Titanium does not.

Is this still an issue ? If so, what exactly is the issue ?

I took a quick look yesterday and a warrior with the SoF client seemed to be regnerating endurance (the number the client showed didn't exactly match #showstats, but it didn't seem far out), and I didn't see endurance drop on zoning.

[ptarp]

Originally Posted by trevius

2. Auras - The logs show Auras being used seem to work the same as normal buffs.

Just wanted to mention that on live auras actually create a spawn that is persistent for about 6 hours, and all but the shaman's (its stationary) stays with the caster even after zoning. If the aura spawn is up, the buff icon goes to the "Auras" window of the caster along with an "effect" buff that goes to the song window on beneficial auras. Most of them have an area affect that will affect all group members or mobs within 50 to 60 paces, and they get the "effect" song in thier window as well.
Wizard and shaman auras are mainly detrimental (I think), and affect mobs within the radius while mage aura adds a buff to their own pet only, and those don't create an effect buff in the window. Chanter, cleric, and melee auras are positive for the group that range from XP bonus (chanter learner's aura), and AC, or regen.
There's a little more to it, but they definately don't work the same as normal buffs.

[trevius]

Thanks for the info. I had never used them myself. The only aura thing I have played with on Live was some freebie veteran reward I had on my character. Clicking it showed an effect around my character, but the buff itself showed up in the normal buff window. I just assumed that was the same thing as the new aura system, but I guess it isn't.

I updated the original post in this thread with a small note about what you mentioned. I also updated and moved the Azone2 and Max Level issues into the recently resolved section, as Derision and KLS (and I think Secrets) have fixed them.