Help Setting Configurable Joint Target Rotation to a child transform's rotation

I'm trying to set a Configurable Joint's target rotation to match a child transform of the object. If the rotation is all zero on the child transform it will always rotate to the same rotation, if the rotation values of the child transform are changed in the editor the rotation is always different when the joint is created. I'm using this for a VR project to grab and using this https://gist.github.com/mstevenson/4958837 to help set the target rotation. Below is my code.

     private void CreateJoint(VRHand hand) 
     {
         if (Joint != null)
             return;
         Joint = gameObject.AddComponent<ConfigurableJoint>();
         Joint.autoConfigureConnectedAnchor = false;
         Joint.xMotion = Joint.yMotion = Joint.zMotion = ConfigurableJointMotion.Locked;
         Joint.angularXMotion = Joint.angularYMotion = Joint.angularZMotion = ConfigurableJointMotion.Free;
         Joint.anchor = m_grabPointAnchor.localPosition;
         Joint.connectedAnchor = Vector3.zero;
         ConfigurableJointExtensions.SetTargetRotationLocal(Joint, Quaternion.Inverse(Quaternion.Inverse(hand.transform.rotation) * m_grabPointAnchor.rotation), m_startRot);
         Joint.connectedBody = hand.rb;
         Joint.enablePreprocessing = false;
         Joint.rotationDriveMode = RotationDriveMode.Slerp;
         slerpDrive.positionSpring = m_PositionSpring;
         slerpDrive.positionDamper = m_PositionDamper;
         slerpDrive.maximumForce = m_MaximumForce;
         Joint.slerpDrive = slerpDrive;
     }

I'm setting the start rotation in the start method

     private void Start()
     {
         m_startRot = m_grabPointAnchor.localRotation;
     }