Paul has a succint online note on vectors and matrices. Vectors are basically a subset of matrices in that they are single row or single column matrices.
Matrices are defined as rectangular arrays, which limits matrices to a maximum of two dimensions. There are several types of special matrices subclasses, including (as mentioned above) single dimension matrices or vectors, square matrices (same number of rows and columns), and others.
Paul’s notes also summarizes matrices operations very well. Wikipedia further provides greater breadth and depth to a summary of the subject.
Multi-dimensional arrays are called tensors. They have a different set of operations, subclasses and other details although most are extensions of the same concept for matrices.
Vectors and matrices are used heavily in neural network computations.