Which way did you go, (1) not use pointers from child to parent (functional programming approach), or (2) kept using such pointers but made it work? Your first paragraph sounds like (1), but the second sounds like (2).
If it was (2), it sounds like you made it work without unsafe code? Can you share some code?
If it was (2), it sounds like you made it work without unsafe code? Can you share some code?