Starting Work on SoF Opcodes/Structs

[trevius]

I think you have been spending too much time around Mort. You are starting to post like him and not making any sense :P

[Secrets]

Originally Posted by trevius

I think you have been spending too much time around Mort. You are starting to post like him and not making any sense :P

I think i'm going to start crying in a corner now -- feel free to delete our three posts to remove clutter, i'm not sure what I was smoking at that time if you know what I mean~

[AndMetal]

Originally Posted by trevius

I went ahead and stuck what I have so far on the SVN. It is pretty sloppy, but it is the point I am at. I made new SoF patch files for it in the common/patches directory. As I fix more things, I will just continue to add the updates to the SVN when I think something is pretty accurate and complete.

I would actually recommend creating a branch, like I did with loading the spells via the database. That way, you can upload changes that may sorta work so that others can work on it also, but won't affect the trunk ("stable", if you want to call it that). Then, once it's a little more stable/accurate, we can update the branch back into the trunk.

[KLS]

Given that the patch files are pretty much self contained I don't think it really needs a new branch. At least not unless you need to modify core parts of the server to make it all work, in which case yes I'd agree with a SoF branch.

[trevius]

Any modifications of the core files that are needed shouldn't affect the other clients. So far, the only core file changes I have made are to add a new opcode. The only other change I currently have planned for core files is to add a second way to handle the sendexpzonein opcode. Since Titanium sends the same op and expects it right back, the current files work fine for Titanium. They also work for SoF, but the order in the EQ Debugs is a bit off, which might be a concern. So, I plan to copy the function in client_packet.cpp for sendexpzonein and make a different version for SoF. That won't effect how the other clients work, only how it handles SoF logins. Basically, instead of sending the sendexpzonein and getting it back from the client, SoF will send the send the sendexpzonein and send back the new worldobjectssent opcode and handle it by sending that same opcode back to the client again.

Sorry if that sounds a little confusing, but I have it tested on my test box and it works fine and doesn't change anything about how the other clients currently work. At this time, I don't think another branch is needed really. I will just leave it set so you have to uncomment a #define to enable using/testing SoF. So, by default, servers won't be able to use it unless they make that change. This is really only because of the possible crash issues with the SoF client that should probably be avoided for now on non-test servers.

As KLS said, since the patch files are handled in their own separate way, the SVN updates I make for SoF should be ok.

Right now, I am still working on and testing some structure changes to see if I can get more stuff working. I think figuring out the clientupdate structure is an important one to focus on for now. It is a pretty small struct (only 36 in size), but it uses some split signed/unsigned blocks that I don't really understand yet.

Here is the structure of it that I am currently working on:

struct PlayerPositionUpdateClient_Struct_WIP //from Client Packet Breakdown and testing
{
/*0000*/ uint16 spawn_id;			// Player's spawn id - Verified!
/*0002*/ uint16	sequence;			//increments one each packet - Verified!
/*0004*/ float delta_heading;		// Change in heading - Looks Accurate
/*0008*/ float camera_angle;		//unused camera incline angle
/*0012*/ float x_pos;				// x coord - Verified!
/*0016*/ float heading;				//Heading - Looks Accurate
/*0020*/ signed z_pos:10;			//Not Positive
		 signed y_pos:10;			//Not Positive
		 unsigned padding1:12;
/*0024*/ float delta_z;				// Change in z - Looks Accurate
/*0028*/ signed delta_y:20;			//Not Positive
		 unsigned padding2:12;		// ***Placeholder
/*0032*/ signed delta_x:20;			//Not Positive
		 unsigned animation:12;		//Not Positive
/*0036*/ 
};

The actual packet example from that looks like this:

3F 00 24 3B 00 00 00 00 - 00 40 9a 00 d7 15 a8 c0
23 01 00 00 9a e0 a2 c0 - 00 00 00 00 00 00 00 00
00 00 00 00

Really, once the spawn struct and clientupdate structs are worked out, I think that should have most of the struct work done. I am sure there will still be more little things to work out in the other structs, but after that, it will mostly be finding missing opcodes. I still haven't worked out a way to find all missing opcodes just yet, but I have been verifying more new ones almost every day.

If I only had a packet collect from 9/7/2007 or shortly after that, it would make this whole process ALOT easier lol.

[KLS]

That's not really what I was talking about, that all really falls under patch files. For example when we wanted to fix inter-zoning we had to actually modify how you zone as 6.2 sent a different series of packets overall than titanium and so required changes to both the patch files and the zone server. When I talk about "core changes" that's things that aren't contained to a certain version and could potentially affect other versions. =p

[Yeormom]

Just do us a flavor and make SoF commits contain only SoF related changes!

[trevius]

As if I had time to make any other changes while I am working on SoF. This thing is like a full time job that I don't get paid for lol.

I will probably be doing an SVN commit for SoF tonight. There is alot more progress since the initial release. Spawns now show up (missing names though), you can actually move around in the zone instead of being stuck in place, doors spawn and ground spawns spawn (but still need work), and a few new opcodes are verified.

Items are a big one that needs work, but those shouldn't be too bad once I know how to do the serialization like a structure so it sends it the way it needs to. I think it is almost to the point where other people might actually be able to start helping out on it soon. I am probably going to try to write a beginner's guide in the wiki for packet structures and maybe some opcode stuff.

[Yeormom]

Originally Posted by trevius

This thing is like a full time job that I don't get paid for lol.

You just described the entire EQemu project!

I am probably going to try to write a beginner's guide in the wiki for packet structures and maybe some opcode stuff.

I do so enjoy documentation~

[trevius]

Back to working on the item serialization again, I broke down items into as close to an exact structure as I can figure out. This is for EQLive, but no new fields have changed from SoF to Live as far as I know, so it should also apply to SoF. This structure may not be 100%, but I think it is at least 95% accurate. This is based off of what I could find from 13th floor and packet collects from EQLive.

struct ItemSerialization_Struct {

/*0000*/	uint32 stacksize;
/*0004*/	uint32 unknown004; // 00 00 00 00
/*0008*/	uint32 price;
/*0012*/	uint32 slot;
/*0016*/	uint32 merchcount;
/*0020*/	uint32 unknown020; // 00 00 00 00
/*0024*/	sint32 serialnumber;
/*0028*/	uint32 instnodrop;
/*0032*/	uint32 typepotion;
/*0036*/	uint32 unknown036; // 00 00 00 00
/*0040*/	uint32 unknown040; // 00 00 00 00
/*0044*/	uint32 unknown044; // 00 00 00 00
/*0048*/	uint32 unknown048; // 00 00 00 00
/*0052*/	uint32 unknown052; // 00 00 00 00
/*0056*/	sint32 unknown056; // 00 00 00 01
/*0060*/	Item_Struct item;
/*0000*/	uint8	unknown00x1;		// 00 Break for separating augments within items?
/*0000*/	Item_Struct augments;		// Bag Slots/Augments within an item
/*0000*/	uint8	unknown00x2;		// 00 Break for separating items?	

}


/*
** Child struct of Item_Struct:
**	Effect data: Click, Proc, Focus, Worn, Scroll
**
*/
struct ItemClickEffect_Struct {
/*0000*/	sint32	Effect;
/*0004*/	uint32	Type;
/*0008*/	uint8	Level2;
/*0009*/	uint8	Level;
/*0010*/	sint32	MaxCharges;		// Max Charges of this effect
			union {
/*0014*/		uint16  Fulfilment;	// Food fulfilment (How long it lasts)
/*0016*/		sint16  CastTime;	// Cast Time for clicky effects, in milliseconds
			};
/*0018*/	uint32	RecastDelay;		// Delay on Recast in seconds
/*0022*/	uint32	RecastType;		// Recast Type from -1 to 18
/*0026*/	uint32	unknown0005;
/*0030*/	char	effectname[1];		//click name - Default is NULL
/*0031*/	uint32	unknown0007;
/*0035*/
};

struct ItemProcEffect_Struct {
/*0000*/	sint32	Effect;
/*0004*/	uint32	Type;
/*0008*/	uint8	Level2;
/*0009*/	uint8	Level;
/*0010*/	uint32	unknown0001;
/*0014*/	uint32	unknown0002;
/*0018*/	uint32	unknown0003;
/*0022*/	uint32	unknown0004;
/*0026*/	sint32	ProcRate;		// Proc Rate - 100 is default
/*0030*/	char	effectname[1];		//proc name - Default is NULL
/*0031*/	uint32	unknown0007;
/*0035*/
};

struct ItemWornFocusScrollEffect_Struct {
/*0000*/	sint32	Effect;
/*0004*/	uint32	Type;
/*0008*/	uint8	Level2;
/*0009*/	uint8	Level;
/*0010*/	uint32	unknown0001;
/*0014*/	uint32	unknown0002;
/*0018*/	uint32	unknown0003;
/*0022*/	uint32	unknown0004;
/*0026*/	uint32	unknown0005;
/*0030*/	char	effectname[1];		//focus, worn, or scroll name - Default is NULL
/*0031*/	uint32	unknown0007;
/*0035*/
};

struct ItemAugmentSlot_Struct {
/*0000*/	uint8	AugSlotType;		// LDoN: Augment Slot 1-5 Type
/*0001*/	sint32	AugSlotVisible;		// LDoN: Augment Slot 1-5 Unknown
/*0005*/	uint8	AugSlotUnk2;		// LDoN: Augment Slot 1-5 Unknown
/*0006*/
};	

#define MAX_AUGMENT_SLOTS 5

struct Item_Struct {
	// Non packet based field
	// uint8	MinStatus; 
	// bool	LoreFlag;		// This will be true if LoreGroup is non-zero
	// bool	SummonedFlag;		// Unused in SoF

	// Packet based fields
	// uint8	ItemClass;		// Item Type: 0=common, 1=container, 2=book
	char	Name[1];		// Variable Length String for Name
	uint8	unknown0001;		// 00 Break between Strings
	char	Lore[1];		// Variable Length String for Item Lore Text
	uint8	unknown0002;		// 00 Break between Strings
	char	IDFile[1];		// Variable Length String for Visible Model Number - IT63
	uint8	unknown0003;		// 00 Break between Strings
	uint32	ID;			// Unique ID (also PK for DB)
	uint8	Weight;			// Item weight * 10
	uint8	NoRent;			// No Rent: 0=norent, 255=not norent
	uint8	NoDrop;			// No Drop: 0=nodrop, 255=not nodrop
	uint8	Size;			// Size: 0=tiny, 1=small, 2=medium, 3=large, 4=giant
	uint8	Slots;			// Bitfield for which slots this item can be used in
	char	Price[8];		// Item cost (?) maybe int64?
	uint32	Icon;			// Icon Number
	uint8	unknown0013;		// UNK013 - 01
	uint32	unknown0014;		// UNK014 - One of these are probably int32	
	bool	BenefitFlag;		// Does this have a benefit flag?
	bool	Tradeskills;		// Is this a tradeskill item?
	sint8	CR;			// Save vs Cold
	sint8	DR;			// Save vs Disease
	sint8	PR;			// Save vs Poison
	sint8	MR;			// Save vs Magic
	sint8	FR;			// Save vs Fire
	sint8	Corruption;		// New Save vs Corruption ****
	sint8	AStr;			// Strength
	sint8	ASta;			// Stamina
	sint8	AAgi;			// Agility
	sint8	ADex;			// Dexterity
	sint8	ACha;			// Charisma
	sint8	AInt;			// Intelligence
	sint8	AWis;			// Wisdom
	sint32	HP;			// HP
	sint32	Mana;			// Mana	
	uint32	Endur;			// Endurance
	sint32	AC;			// AC	
	uint32	unknown0015;		// 
	uint32	unknown0016;		// 
	uint32	unknown0017;		// 
	uint32	Classes;		// Bitfield of classes that can equip item (1 << class#)
	uint32	Races;			// Bitfield of races that can equip item (1 << race#)
	uint32	Deity;			// Bitmask of Deities that can equip this item
	sint32	SkillModValue;		// % Mod to skill specified in SkillModType
	uint32	unknown0038;		// UNK038 - Default is 0
	uint32	SkillModType;		// Type of skill for SkillModValue to apply to
	uint32	BaneDmgRace;		// Bane Damage Race
	uint32	BaneDmgBody;		// Bane Damage Body
	uint32	BaneDmgRaceAmt;		// Bane Damage Race Amount
	sint32	BaneDmgAmt;		// Bane Damage Body Amount
	bool	Magic;			// True=Magic Item, False=not
	sint32	CastTime_;		// Cast Time in Milliseconds
	uint8	ReqLevel;		// Required Level to use item
	uint8	RecLevel;		// Recommended level to use item
	uint8	RecSkill;		// Recommended skill to use item (refers to primary skill of item)
	uint32	BardType;		// Bard Skill Type
	sint32	BardValue;		// Bard Skill Amount
	sint8	Light;			// Light
	uint8	Delay;			// Delay * 10
	uint32	ElemDmgType;		// Elemental Damage Type (1=magic, 2=fire)
	uint32	ElemDmgAmt;		// Elemental Damage
	uint32	Range;			// Range of item
	uint32	Damage;			// Delay between item usage (in 0.1 sec increments)
	uint32	Color;			// RR GG BB 00 <-- as it appears in pc
	uint32	ItemType;		// Item Type/Skill (itemClass* from above)
	uint32	Material;		// Item material type
	uint32	unknown0060;		// UNK060 - Default is 0
	uint8	unknown0061;		// UNK061 - Default is 0
	float	SellRate;		// Sell rate
	sint32	CombatEffects;		// PoP: Combat Effects +
	sint32	Shielding;		// PoP: Shielding %
	sint32	StunResist;		// PoP: Stun Resist %
	sint32	StrikeThrough;		// PoP: Strike Through %
	uint32	ExtraDmgSkill;		// PoP: Extra Damage Skill
	uint32	ExtraDmgAmt;		// PoP: Extra Damage Amount
	sint32	SpellShield;		// PoP: Spell Shield %
	sint32	Avoidance;		// PoP: Avoidance +
	sint32	Accuracy;		// PoP: Accuracy +
	uint32	CharmFileID;		// ID of the Charm File
	sint32	FactionMod1;		// Faction Mod 1
	sint32	FactionAmt1;		// Faction Amt 1
	sint32	FactionMod2;		// Faction Mod 2
	sint32	FactionAmt2;		// Faction Amt 2
	sint32	FactionMod3;		// Faction Mod 3
	sint32	FactionAmt3;		// Faction Amt 3
	sint32	FactionMod4;		// Faction Mod 4
	sint32	FactionAmt4;		// Faction Amt 4
	char	CharmFile[1];		// Name of the Charm File
	uint8	unknown0062;		// 00 Break between Strings
	uint32	AugType;
	uint16	AugRestrict;
	uint16	AugDistiller;
	ItemAugmentSlot_Struct augslots[MAX_AUGMENT_SLOTS]; //Augment Slots
	uint32	PointType;
	uint32	LDoNTheme;
	uint32	LDoNPrice;
	uint32	unknown0098;		// UNK098
	uint32	LDoNSold;
	uint8	BagType;		// 0:Small Bag, 1:Large Bag, 2:Quiver, 3:Belt Pouch ... there are 50 types
	uint8	BagSlots;		// Number of slots: can only be 2, 4, 6, 8, or 10
	uint8	BagSize;		// 0:TINY, 1:SMALL, 2:MEDIUM, 3:LARGE, 4:GIANT 
	uint8	BagWR;			// 0->100
	uint8	Book;			// 0=Not bool, 1=Book
	uint16	BookType;
	uint32	Filename;		// Filename for book data
	uint8	LoreGroup;		// LoreGroup seems to have replaced LoreFlag
	bool	ArtifactFlag;		// Flag as an Artifact
	bool	PendingLoreFlag;	// ???  Maybe just unknown?
	uint32	Favor;			// Individual favor
	uint32	GuildFavor;		// Guild favor
	uint8   FVNoDrop;		// Firiona Vie nodrop flag
	uint8	DotShielding;		// May need to be moved to the PoP stuff ****
	uint32	Attack;
	uint32	Regen;
	uint32	ManaRegen;
	uint32	EnduranceRegen;
	uint8	Haste;
	uint8	DamageShield;
	uint32	unknown0120;		// UNK120 - Default is -1 	
	uint8	unknown0121;		// UNK121 - Default is 0 		
	bool	Attuneable;
	bool	NoPet;
	uint16	unknown0124;		// UNK124 - Default 0
	bool	PotionBelt;
	uint32	PotionBeltSlots;
	uint8	StackSize;
	bool	NoTransfer;		//Swapped Positions with StackSize ****
	// bool	Stackable;		//Not in 13th Floor ???
	bool	QuestItemFlag;
	uint8	unknown0131;		// UNK131 - Default is 0	
	uint8	unknown0132[19];		// UNK132 - Default is 0	
	ItemClickEffect_Struct Click;
	ItemProcEffect_Struct Proc;
	ItemWornFocusScrollEffect_Struct Worn, Focus, Scroll;
	uint32	unknown0193;//UNK193 - Default is 0
	uint8	purity;			//purity - Newly Added - Default is 0, but some go up to 75
	uint8	dsmitigation;		//dsmitigation - Newly Added - Default is 0, but some are up to 2
	uint8	herostr;		//heroic_str - Newly Added - Default is 0
	uint8	heroint;		//heroic_int - Newly Added - Default is 0
	uint8	herowis;		//heroic_wis - Newly Added - Default is 0
	uint8	heroagi;		//heroic_agi - Newly Added - Default is 0
	uint8	herodex;		//heroic_dex - Newly Added - Default is 0
	uint8	herosta;		//heroic_sta - Newly Added - Default is 0
	uint8	herocha;		//heroic_cha - Newly Added - Default is 0
	uint32	heropoison;		//HeroicSvPoison - Newly Added - Default is 0
	uint32	heromagic;		//HeroicSvMagic - Newly Added - Default is 0
	uint32	herofire;		//HeroicSvFire - Newly Added - Default is 0
	uint32	herodisease;		//HeroicSvDisease - Newly Added - Default is 0
	uint32	herocold;		//HeroicSvCold - Newly Added - Default is 0
	uint32	herocorruption;		//HeroicSvCorruption - Newly Added - Default is 0
	uint32	healamt;		//healamt - Newly Added - Default is 0, but some are up to 9
	uint32	spelldmg;		//spelldmg - Newly Added - Default is 0, but some are up to 9
	uint32	clairvoyance;		//clairvoyance - Newly Added - Default is 0, but some are up to 10
	uint32	backstabdmg;		//backstabdmg - Newly Added - Default is 0, but some are up to 65
	uint32	evolvinglevel;		//evolvinglevel - Newly Added - Default is 0, but some are up to 7
	uint32	MaxPower;		//MaxPower - Newly Added
	uint32	Power;			//Power - Newly Added

};

Since items are handled in a different way than any other struct, I am not exactly sure how to implement this properly. Currently, the emulator sends items as a long string. For SoF, they need to be sent like any other normal structure. Right now, I can't figure out how to add these structs in without having to mess with the item serialization for other clients. I will keep working on it, but it is a bit above my skill level right now. For starters, I need to figure out how to change the serialization from using MakeAnyLengthString and instead for it to send the actual data as it is written in the struct.

[trevius]

I was able to get Targeting working finally and also able to get names to show up now too. The only other major thing to get working from the spawn structure is size. Most of the rest isn't really required and could be done later at any point. These are a couple of major things that were holding me back from making much more progress, so having those working will help alot to getting the game playable.

The actual process I used to find those settings is a bit of a pain, but seems to actually work fairly well. Basically, I took all of the unknowns in the spawn structure and have been setting each one to send 1s for all fields in them. So, by doing this for each section of unknowns 1 at a time, I can compile the changes and then start the server and log in to observe if there were any differences and what they are.

Here is the actual spawn encoding that I am using to test:

ENCODE(OP_NewSpawn) {  ENCODE_FORWARD(OP_ZoneSpawns); }
ENCODE(OP_ZoneEntry){  ENCODE_FORWARD(OP_ZoneSpawns); }
ENCODE(OP_ZoneSpawns) {
	//consume the packet
	EQApplicationPacket *in = *p;
	*p = NULL;
	
	//store away the emu struct
	unsigned char *__emu_buffer = in->pBuffer;
	Spawn_Struct *emu = (Spawn_Struct *) __emu_buffer;
	
	//determine and verify length
	int entrycount = in->size / sizeof(Spawn_Struct);
	if(entrycount == 0 || (in->size % sizeof(Spawn_Struct)) != 0) {
		_log(NET__STRUCTS, "Wrong size on outbound %s: Got %d, expected multiple of %d", opcodes->EmuToName(in->GetOpcode()), in->size, sizeof(Spawn_Struct));
		delete in;
		return;
	}
	
	//make the EQ struct.
	in->size = sizeof(structs::Spawn_Struct)*entrycount;
	in->pBuffer = new unsigned char[in->size];
	structs::Spawn_Struct *eq = (structs::Spawn_Struct *) in->pBuffer;
	
	//zero out the packet. We could avoid this memset by setting all fields (including unknowns)
	//in the loop.
	memset(in->pBuffer, 0, in->size);
	
	//do the transform...
	int r;
	int k;
	for(r = 0; r < entrycount; r++, eq++, emu++) {

		eq->deity = emu->deity;
		eq->gender = emu->gender;
		for(k = 0; k < 9; k++) {
			eq->equipment[k].equip0 = emu->equipment[k];
			eq->equipment[k].equip1 = 0;
			eq->equipment[k].itemId = 0;
			eq->colors[k].color = emu->colors[k].color;
		}
		eq->guildID = emu->guildID;
		eq->class_ = emu->class_;
		eq->gm = emu->gm;
		eq->runspeed = emu->runspeed;
		eq->light = emu->light;
		eq->level = emu->level;
		eq->race = emu->race;
		strcpy(eq->suffix, emu->suffix);
		eq->bodytype = emu->bodytype;
		eq->curHp = emu->curHp;
		strcpy(eq->lastName, emu->lastName);
		strcpy(eq->title, emu->title);
		eq->NPC = emu->NPC;
		eq->x = emu->x;
		eq->deltaX = emu->deltaX;
		eq->deltaY = emu->deltaY;
		eq->z = emu->z;
		eq->deltaHeading = emu->deltaHeading;
		eq->y = emu->y;
		eq->deltaZ = emu->deltaZ;
		eq->animation = emu->animation;
		eq->heading = emu->heading;
		eq->spawnId = emu->spawnId;
		strcpy(eq->name, emu->name);
		eq->petOwnerId = emu->petOwnerId;
		eq->anon = emu->anon;
		eq->walkspeed = emu->walkspeed;

		eq->targetable = 1; //New Field - Force NPCs to Targetable for now
		eq->showname = 1; //New Field - Toggles Name Display on or off - 0 = off, 1 = on
		eq->linkdead = 0; //New Field - Toggles LD on or off after name - 0 = off, 1 = on

		//Hack Test for finding more fields in the Struct:
		memset(eq->unknown0001, 0x01, sizeof(eq->unknown0001));
		memset(eq->unknown0005, 0x01, sizeof(eq->unknown0005)); // 15
		//memset(eq->unknown0008, 0x01, sizeof(eq->unknown0008)); // 13
		//memset(eq->unknown0048, 0x01, sizeof(eq->unknown0048)); // 12 - No Visible Change?
		//eq->unknown0820 = 1;	//Stand State - Stand/Sit/Crouch
		//eq->unknown0059 = 1; // 1 Turned off on 6 - west bug?
		//memset(eq->unknown0074, 0x01, sizeof(eq->unknown0074)); // 16 - No Visible Change?
		//memset(eq->unknown0077, 0x01, sizeof(eq->unknown0077));
		//memset(eq->unknown0079, 0x01, sizeof(eq->unknown0079));
		//memset(eq->unknown0106, 0x01, sizeof(eq->unknown0106)); // 11 - No Visible Change?
		//memset(eq->unknown0107, 0x01, sizeof(eq->unknown0107));
		//memset(eq->unknown0110, 0x01, sizeof(eq->unknown0110));
		//eq->unknown0111 = 1; // 1 - No Visible Change?
		//eq->unknown0613 = 0; //was bodytype
		//memset(eq->unknown0154, 0x01, sizeof(eq->unknown0154)); // 2 - freeze in place?
		//memset(eq->unknown0263, 0x01, sizeof(eq->unknown0263)); // 1 - no player character visible?
		//memset(eq->unknown0281, 0x01, sizeof(eq->unknown0281)); // 2 3
		//eq->unknown0307 = 1; // 9 10 11 - No Visible Change?
		//memset(eq->unknown0308, 0x01, sizeof(eq->unknown0308)); //
		//memset(eq->unknown0309, 0x01, sizeof(eq->unknown0309)); // 8 - No Visible Change?
		//memset(eq->unknown442, 0x01, sizeof(eq->unknown442)); // 6 - crash?
		//eq->unknown443 = 1; // 1 turned off on 9 - No Visible Change?
		//memset(eq->unknown0760, 0x01, sizeof(eq->unknown0760)); // 4 avatar height?
		//eq->unknown0779 = 0; // 1 - int32 avatar height?
		//memset(eq->unknown0496, 0x01, sizeof(eq->unknown0496)); // 4 5

		// 1 all set to one shows you at floor level
		// 2 seems to cause invis and freeze in place and west bug
		// 3 causes no player character, but spawns work and has west bug
		// 4 looks just like 2 accept you can move and are at floor level
		// 5 Looks like 3
		// 6 Caused a crash
		// 7 west bug gone 
		// 8 Looks normal accept no player character
		// 9 Targetable mobs!!! Still no player character...
		// 10 Narrowing down bodytype
		// 11 Non-targetable again and narrowing down bodytype
		// 12 Bodytype location identified!!!  Targeting works...  Nother other visible changes
		// 13 Can target self but not spawns.  West bug again
		// 14 Names showing and also LD showing!!!  Still West Bug though...
		// 15 Show Names field now identified.  

	}
	
	
	//kill off the emu structure and send the eq packet.
	delete[] __emu_buffer;
	
	_log(NET__ERROR, "Sending zone spawns");
	_hex(NET__ERROR, in->pBuffer, in->size);
	
	dest->FastQueuePacket(&in, ack_req);
}

The section noted in blue are the new fields I found and am forcing them to send all spawns with a certain setting in that field. The section noted in green are the actual settings to test the unknown fields. I just uncomment the line of the unknown section I want to test. There are some notes next to some of them as to what I noticed when testing them. There are also numbers next to some as well, which correspond to the section in orange below that. Basically, I tried picking the unknowns that were smallest first. I was doing more than 1 at a time to reduce the number of times I would have to recompile. The numbers tell which ones I turned on for each test number I ran. So, the ones that have 1 next to them were the first ones I tried and the notes in orange below are what I saw as a result from that first test. The notes could have been done better, but at least it was something to reference to help keep track of what was being done and being seen.

The reason I am doing a little write-up on how I did this is so others may be able to use the same technique in the future to help refine the structures even further. Other than being able to figure out the structure from reading the IDA output, which is well above my skill level, this guess and test technique is the only other way I could think of to figure out these needed structure fields.

The same technique should be able to work fine for figuring out the rest of the fields. I think it should be fairly easy to figure out stuff like beard and beard color, face, etc with this technique. So, almost anyone could help refine this list of field positions in the structure.

I will most likely be doing a write-up wiki page on what I know about working with packet structures. It really isn't all too complex, but it is important to understand a few key things that took me a while to figure out on my own.

I will probably do an SVN update with the new targeting and name stuff tonight, or very soon. It is finally getting to the point where it is almost playable. I think getting items working is probably one of the biggest things left to do. I have the item structure worked out, but may need help in finding the best way to implement it.

I was finally able to try attacking a mob for the first time in SoF, and of course it caused EQ to crash :P I am pretty sure the crash was due to the fact that items hadn't been loaded though.

For a while, I was a bit concerned that we would never be able to get SoF functioning well enough to actually use as an upgrade to Titanium. Now, I am fairly confident that all possible issues can be resolved and that it will almost certainly be completed and at least as functional as Titanium is now. It is all really just a matter of time and work. I am really hoping that once I have the game somewhat playable and most systems functioning, that others will start assisting in the project. I know most people think this stuff is well beyond their skill level, but if I can learn it, most people with technical skill can as well. And, I think that if I write a couple of wiki pages to help people understand what to do, it will make a huge difference in who can start helping.

[Yeormom]

So much like myself, you too have little room for your other hobbies. :(

[cavedude]

Originally Posted by trevius

Back to working on the item serialization again, I broke down items into as close to an exact structure as I can figure out. This is for EQLive, but no new fields have changed from SoF to Live as far as I know, so it should also apply to SoF. This structure may not be 100%, but I think it is at least 95% accurate. This is based off of what I could find from 13th floor and packet collects from EQLive.

Just wanted to remind you that AugSlotUnk; is now known as AugSlotVisible.

[trevius]

Thanks. Yeah, I noticed that from 13th floor back when I first started working on serialization. I went ahead and changed it in the structure I posted above so it should be accurate now.

[trevius]

Using the same method I described before, I was able to find the helm field in the spawn structure and also in the illusion structure last night. I think this method could even be useful for finding more stuff in Titanium if we needed it. Even though it is a bit of a pain, it isn't really too hard to do. It just takes time.

I also wanted to mention that I was changing through races last night and noticed that there are considerably more global races in SoF than there are in Titanium. There are a few new races as globals that aren't in Titanium at all, including a dragon model with 5 textures! And also many of the models that exist in Titanium are now set as globals as well like Shilskins, all Pirates (LoY) and at least a few others. This is great for custom servers that want to use a variety of races in custom zones and don't want to require any changes on the client side. I also saw some new global races that were objects. One was a big floating skull with flies and a centipede crawling around it and some of the others looked like runes and floating things. The new races look really cool from what I have seen so far. I am really excited to be able to start making content using them.

The last major thing I want to find from the spawn structure is model size. Other than that, I do need to figure out why texture doesn't seem to work either. I don't see texture as one of the fields in Titanium, so I am guessing that it comes from the appearance packet instead of from the spawn struct.

Items and spawn/client position updates are still the 2 most important things that need to be done right now. Once those 2 are complete, I think the rest will be just filling in the little stuff and working out the details to refine it. Once I can fight and kill an NPC, I think it will be ready for player beta testing. The only things that I think are holding that up right now are the items and position updates I already mentioned. Basically, things are getting really close to being playable.

If anyone is interested in helping, let me know. I could use help from someone with more experience in packet structures. I don't quite understand what unions are. Most importantly, I don't understand how signed and unsigned fields work. Here is an example of the client position update structure from Titanium and it uses signed and unsigned, which seems to combine a into an int32 somehow, but I don't understand exactly how...

struct PlayerPositionUpdateClient_Struct
{
/*0000*/ uint16	spawn_id;
/*0022*/ uint16	sequence;	//increments one each packet
/*0004*/ float y_pos;                 // y coord
/*0008*/ float delta_z;            // Change in z
/*0016*/ float delta_x;            // Change in x
/*0012*/ float delta_y;            // Change in y
/*0020*/ sint32 animation:10,     // animation
         		delta_heading:10,  // change in heading
         		padding0020:12;   // ***Placeholder (mostly 1)
/*0024*/ float x_pos;                 // x coord
/*0028*/ float z_pos;                 // z coord
/*0034*/ uint16 heading:12,     // Directional heading
         		padding0004:4;  // ***Placeholder
/*0032*/ uint8 unknown0006[2];  // ***Placeholder
/*0036*/
};

That struct is pretty straight foward, but I don't understand the sections in green and how those actually work. The client update comes from the client, so it is easy to look at them and see what is being changed and what the order is. I just don't know how to figure out how to write those green section types.

Here is the struct I am working on for SoF so far:

struct PlayerPositionUpdateClient_Struct_WIP //from Client Packet Breakdown and testing
{
/*0000*/ uint16 spawn_id;			// Player's spawn id - Verified!
/*0002*/ uint16	sequence;			//increments one each packet - Verified!
/*0004*/ float delta_heading;		// Change in heading - Looks Accurate
/*0008*/ float camera_angle;		//unused camera incline angle
/*0012*/ float x_pos;				// x coord - Verified!
/*0016*/ float heading;				//Heading - Looks Accurate
/*0020*/ signed z_pos:10;			//Not Positive
		 signed y_pos:10;			//Not Positive
		 unsigned padding1:12;
/*0024*/ float delta_z;				// Change in z - Looks Accurate
/*0028*/ signed delta_y:20;			//Not Positive
		 unsigned padding2:12;		// ***Placeholder
/*0032*/ signed delta_x:20;			//Not Positive
		 unsigned animation:12;		//Not Positive
/*0036*/ 
};

As you can see, I only have a small bit of that structure verified. I am really just guessing with the sections in green since I have no clue how to figure that part out yet. If someone can explain how those signed and unsigned with sizes work, that may help. All I know is that they all have to add up to 32 for each section of them.

Unfortunately, the positioning stuff is one of the few things that SoE seems to completely switch around almost every time they patch. The only good thing is that it is a fairly small packet, so even just guessing, it shouldn't be way too hard to get it set correctly.

I will probably try to get an update onto SVN tonight. I know there isn't much reason to update it regularly for SoF changes at this point, but when I make major breakthroughs, I think it is worth it. The main reason I am doing the updates at all is that hopefully at some point, it will be working enough that someone else might realize that they could start helping so we could get this expansion done much quicker. I think once people start realizing that SoF will actually get completed, they might be more willing to put some of their own time into helping. Up until recently, I think it was questionable if it would ever work or not, but at this point, I am very confident that it will be able to fully replace Titanium (though Titanium will still be usable of course).

I still don't have an ETA on when it will be playable or widely used on servers, but it is getting close to time to maybe consider picking up that last copy of SoF at your local retailer if you happen to see a copy and have an extra 20$ laying around that you don't mind spending :P At least anyone interested in testing or helping out with the SoF development should definitely get a copy if they find one. Of course, don't blame me if for some freak reason it never gets finalized!