Designing adaptable control laws that can transfer between different robots\nis a challenge because of kinematic and dynamic differences, as well as in\nscenarios where external sensors are used. In this work, we empirically\ninvestigate a neural networks ability to approximate the Jacobian matrix for an\napplication in Cartesian control schemes. Specifically, we are interested in\napproximating the kinematic Jacobian, which arises from kinematic equations\nmapping a manipulator's joint angles to the end-effector's location. We propose\ntwo different approaches to learn the kinematic Jacobian. The first method\narises from visual servoing where we learn the kinematic Jacobian as an\napproximate linear system of equations from the k-nearest neighbors for a\ndesired joint configuration. The second, motivated by forward models in machine\nlearning, learns the kinematic behavior directly and calculates the Jacobian by\ndifferentiating the learned neural kinematics model. Simulation experimental\nresults show that both methods achieve better performance than alternative\ndata-driven methods for control, provide closer approximations to the proper\nkinematics Jacobian matrix, and on average produce better-conditioned Jacobian\nmatrices. Real-world experiments were conducted on a Kinova Gen-3 lightweight\nrobotic manipulator, which includes an uncalibrated visual servoing experiment,\na practical application of our methods, as well as a 7-DOF point-to-point task\nhighlighting that our methods are applicable on real robotic manipulators.\n