Skip to content

Matrix Manipulation and Data Preparation

Financial datasets often need to be reshaped or combined before analysis. NumPy includes tools for concatenating arrays, stacking data vertically or horizontally, reshaping matrices, repeating values, inserting rows or columns, and padding arrays.

These operations are especially important when preparing data for machine learning, aligning time-series data, combining multiple data sources, or restructuring arrays into whatever format a particular model or function expects.

Python Code Examples: Matrix Manipulation and Data Preparation

Reshaping arrays

.reshape() lets you rearrange the same underlying data into a different shape, as long as the total number of elements stays the same:

import numpy as np

g = np.arange(24)              # a flat, 1D array of 24 values: [0, 1, 2, ..., 23]

g2 = g.reshape(4, 6)            # rearranged into 4 rows and 6 columns
print(g2)
print("Rank:", g2.ndim)          # .ndim gives the number of dimensions (here, 2)

The total element count must match exactly, 24 values can become a 4x6 array, a 2x12 array, or a 2x3x4 array, but not, for example, a 5x5 array, since that would need 25 values.

# Reshaping can add extra dimensions too, useful for matching what certain functions expect
h = np.arange(5).reshape(1, 1, 5)
print(h)

k = np.arange(6).reshape(2, 3)
print(k)

# A 3D array, made up of two 3x4 matrices stacked together
c = np.arange(24).reshape(2, 3, 4)
print(c)

# The same 48 values reshaped in different ways
b = np.arange(48).reshape(4, 12)     # 4 rows, 12 columns
print(b)

c = b.reshape(4, 2, 6)                # the same 48 values, now viewed as 4 matrices of 2x6
print(c)

A note on .reshape(): reshaping doesn't change the underlying data itself, only how it's organised and viewed. This is why it's such a cheap, fast operation, NumPy isn't copying or recalculating any values, just relabelling how they're grouped.

Stacking Arrays Together

When preparing data from multiple sources, for example combining several assets' return series, it's common to need to join arrays together. NumPy offers several closely related tools for this:

q1 = np.full((3, 4), 1.0)
q2 = np.full((4, 4), 2.0)
q3 = np.full((3, 4), 3.0)

# np.vstack() stacks arrays vertically (adding rows)
# The arrays must have the same number of columns
q4 = np.vstack((q1, q2, q3))
print(q4)                        # shape (10, 4): 3 + 4 + 3 = 10 rows, 4 columns throughout

# np.hstack() stacks arrays horizontally (adding columns)
# The arrays must have the same number of rows
q5 = np.hstack((q1, q3))
print(q5)                        # shape (3, 8): both have 3 rows, so columns are combined: 4 + 4 = 8

# Attempting to hstack arrays with a mismatched number of rows raises an error
try:
    q5 = np.hstack((q1, q2, q3))   # q2 has 4 rows, but q1 and q3 only have 3
except ValueError as e:
    print(e)

# np.concatenate() is the more general tool underneath both vstack and hstack
# axis=0 stacks vertically, equivalent to vstack
q7 = np.concatenate((q1, q2, q3), axis=0)
print(q7)

As a memory aid: vstack adds rows (stacks vertically), hstack adds columns (stacks horizontally), and concatenate is the flexible underlying function that lets you choose the axis explicitly.

Reshaping for Multi-Dimensional Work

r = np.arange(24).reshape(6, 4)     # a 2D array: 6 rows, 4 columns
print(r)

t = np.arange(24).reshape(4, 2, 3)   # a 3D array: 4 matrices, each 2 rows x 3 columns
print(t)

m1 = np.arange(10).reshape(2, 5)     # a 2D array: 2 rows, 5 columns
print(m1)

# Reshaping a 1D array into an explicit 2D "row vector"
m2 = np.arange(5)                     # a plain 1D array of 5 values
m2r = m2.reshape(1, 5)                 # the same values, but now explicitly a single row
print(m2r)

n1 = np.arange(10).reshape(2, 5)
print(n1)

n2 = np.arange(15).reshape(5, 3)
print(n2)

Why reshape into a "row vector" (1, 5) instead of leaving it 1D (5,)? Some operations, particularly matrix multiplication and certain plotting or modelling functions, treat 1D and 2D arrays differently. Explicitly reshaping into a (1, n) or (n, 1) shape removes any ambiguity about whether you mean a row or a column, which becomes especially important once you start combining arrays of different shapes together.

Choosing an Appropriate Data Type

# dtype lets you control how values are stored in memory
# np.uint8 stores unsigned 8-bit integers, i.e. whole numbers from 0 to 255
b = np.arange(24, dtype=np.uint8).reshape(2, 3, 4)
print(b)

Choosing a smaller, more specific data type like np.uint8 can meaningfully reduce memory usage when working with very large datasets, though it comes with a trade-off: it can only store a limited range of values, so it's not suitable for data that might go negative or exceed 255.

A Worked Example: Reshaping Simulated Data

Bringing this together with the simulated data from the previous section, imagine you've generated random return series for three assets and stacked them in two different ways, you can now reshape each result into a cleaner, more usable layout:

np.random.seed(1)
rand_1 = np.random.normal(0, 1, size=(50, 1))
rand_2 = np.random.normal(0, 1, size=(50, 1))
rand_3 = np.random.normal(0, 1, size=(50, 1))

V_Stack = np.vstack((rand_1, rand_2, rand_3))    # shape (150, 1): all three stacked into one long column
H_Stack = np.hstack((rand_1, rand_2, rand_3))     # shape (50, 3): three separate columns, side by side

# Reshaping the vertically stacked data into 50 rows of 3 columns each
# effectively "unstacking" it back into a table with one column per asset
V_Stack_Reshape = V_Stack.reshape(50, 3)
print(V_Stack_Reshape)

# Reshaping the horizontally stacked data into a single long column
# effectively flattening the table into one continuous series
H_Stack_Reshape = H_Stack.reshape(150, 1)
print(H_Stack_Reshape)

Reading this example: V_Stack_Reshape and H_Stack_Reshape end up containing the same 150 values, just organised differently, one as a 50x3 table, the other as a single 150x1 column. Neither is "more correct" than the other; which shape you want depends entirely on what you're about to do with the data next, for example, whether a later function expects one long series or a table with a separate column per asset.

NumPy provides the numerical foundation for many later Python tools. In the next section, we move from numerical arrays to tabular datasets using pandas.