2D Velocity movement on slope hops when moving left/right

I'm working on a 2D platformer. I have run into an issue with straight slopes and left/right movement. I am controlling movement based on velocity to give snappy controls. The problem is that when going down a slope, if you go from 0 to 1 it gives a burst of velocity that makes it look like you hop initially then continue down the slope at a normal rate. It also hops a bit when moving just left.

Here is a video that shows the issue

Here is the relevant code. dirAxis is a 1,0,-1 variable based on my touch input. moveSpeed is a constant int.

  void FixedUpdate ()
     {
         rigid2d.velocity = new Vector2(dirAxis * moveSpeed, rigid2d.velocity.y);
         dirAxis = 0;
     }

A solution I am currently working with is this, and it works except for it's unintended effects on grounding while in the air (e.g. if I hop on a one directional platform I all of a sudden lose the ability to continue going up and jitter through the "ground" collider. This obviously is not the best correct solution, but it does work on the ground.

 void FixedUpdate ()
 {
     Vector2 moveVec = Vector2.zero;
     if (!isInAir)
         moveVec = dirAxis * moveSpeed * transform.right;
     else
         moveVec = new Vector2(dirAxis * moveSpeed, rigid2d.velocity.y);

     rigid2d.velocity = moveVec;
     dirAxis = 0;
 }

Things to note:

-The Unity 2D standard character controller (Robot dude) has this same bug when his movement is set to a high enough speed (~15-20) (also uses velocity movement so I'm not surprised.

-Setting the gravity to a higher setting while grounded will not work for me (already tried it and it messes with my other physics).

-My character is rotating based on the ground he is on, but this issue persists even if his z-axis is 'frozen'