Numpy
Library in Python for numerical computations, data manipulation & mathematical functions
| Feature | Python List | NumPy Array |
| Data type | Any data type | Same data type |
| Mutability | Mutable | Partially mutable (elements can modified, size and shape cannot) |
| Performance | Slower for numerical operations | Faster for numerical operations |
| Memory usage | Higher memory usage | More memory efficient |
| Element wise operation | Loop for element wise operation | Allows vectorized operation |
pip install numpy
Vectorized operation
# Adding 2 to a Python list
python_list = [1, 3, 5, 7, 9]
python_list_with_2 = [x + 2 for x in python_list]
print(“Python List with 2 added:”, python_list_with_2)
Or
az=list(map(lambda x:x+2,python_list))
az
# Adding 2 to a NumPy array
import numpy as np
numpy_array = np.array([1, 3, 5, 7, 9])
numpy_array_with_2 = numpy_array + 2
print(“NumPy Array with 2 added:”, numpy_array_with_2)
o/p:
Python List with 2 added: [3, 5, 7, 9, 11]
NumPy Array with 2 added: [ 3 5 7 9 11]
Performance
import time
# Create large arrays
python_list = list(range(10**6))
numpy_array = np.array(range(10**6))
# Measure the time taken for element-wise addition using Python list
start_time = time.time()
python_list_sum = [x + 2 for x in python_list]
end_time = time.time()
print(f”Time taken for Python list addition: {end_time – start_time} seconds”)
# Measure the time taken for element-wise addition using NumPy array
start_time = time.time()
numpy_array_sum = numpy_array + 2
end_time = time.time()
print(f”Time taken for NumPy array addition: {end_time – start_time} seconds”)
o/p:
Time taken for Python list addition: 0.14735054969787598 seconds
Time taken for NumPy array addition: 0.0025739669799804688 seconds
Memory
import sys
# Create a Python list and a NumPy array with the same data
python_list = list(range(10**6))
numpy_array = np.array(range(10**6))
# Check the size of each object in bytes
size_python_list = sys.getsizeof(python_list)
size_numpy_array = numpy_array.nbytes
print(f”Size of Python list: {size_python_list} bytes”)
print(f”Size of NumPy array: {size_numpy_array} bytes”)
o/p:
Size of Python list: 8000056 bytes
beSize of NumPy array: 4000000 bytes
arange vs linspace
| Feature | np.arange(start,end,step) | np.linspace(start,end,no of element) |
| Specified | Step size | Number of elements |
| Endpoint | Exclusive | Inclusive (by default) |
y = np.arange(10,50, 5)
o/p: array([10, 15, 20, 25, 30, 35, 40, 45])
arr2 = np.linspace(0, 10, 5)
o/p: array([ 0. , 2.5, 5. , 7.5, 10. ])
Creating NumPy arrays
| Method | Syntax | Description | Example |
| From Python list or tuple | np.array(list_or_tuple) | Creates an array from a Python list or tuple | my_list = [1, 2, 3]; my_array = np.array(my_list) |
| Using arange | np.arange(start, stop, step) | Creates an array with evenly spaced values | my_array = np.arange(0, 10, 2) |
| Using linspace | np.linspace(start, stop, num) | Creates an array with evenly spaced values within a specified interval | my_array = np.linspace(0, 1, 5) |
| Using zeros | np.zeros(shape) | Creates an array filled with zeros | my_array = np.zeros((3, 3)) |
| Using ones | np.ones(shape) | Creates an array filled with ones | my_array = np.ones((2, 2)) |
| Using empty | np.empty(shape) | Creates an array with uninitialized values (fast but potentially unsafe) | my_array = np.empty((2, 2)) |
| Using eye or identity | np.eye(n), np.identity(n) | Creates an identity matrix | my_array = np.eye(3) |
| From random numbers | np.random.rand(d0, d1, …), np.random.randn(d0, d1, …), np.random.randint(low, high, size) random_numbers = np.random.randn(3, 3) | Creates arrays with random numbers Rand n creates std normal distribution | random_array = np.random.rand(3, 3) random_numbers = np.random.randn(3, 3) np.random.randint(1,10,5) |
| Full | np.full((2,2),5) – array([[3, 3], [3, 3]]) | ||
| Repeat | np.repeat([1,2,3],2) – array([1, 1, 2, 2, 3, 3]) | ||
| Nditer | a = (np.arange(8)*2) for x in np.nditer(a): print(x, end=’, ‘) |
1D, 2D, 3D arrays
| 1D numpy array | 2D array | 3D array |
| a = np.arange(5, 14, 2) a = np.array([5, 8, 12]) b = np.linspace(5,14,5, dtype=’int’) | arr = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]]) shape = (3, 4) arr = np.zeros(shape) arr = np.ones((3,4)) arr = np.empty(shape) | nested_list = [[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]], [[13, 14, 15, 16], [17, 18, 19, 20], [21, 22, 23, 24]]] arr = np.array(nested_list) shape = (2, 3, 4) arr = np.zeros(shape) rr = np.ones(shape) arr = np.empty(shape) |
| -[ 5 7 9 11 13] -[ 5 8 12] -array([ 5, 7, 9, 11, 14]) | [[ 1 2 3 4] [ 5 6 7 8] [ 9 10 11 12]] | [[[ 1 2 3 4] [ 5 6 7 8] [ 9 10 11 12]] [[13 14 15 16] [17 18 19 20] [21 22 23 24]]] |
Reshape operation
| Operation | Syntax | Description | Example |
| Reshape to 2D | arr.reshape(new_rows, new_cols) | Reshapes a 1D array into a 2D array | arr = np.arange(12); arr.reshape(3, 4) |
| Reshape to 3D | arr.reshape(new_rows, new_cols, new_depth) | Reshapes a 1D or 2D array into a 3D array | arr = np.arange(24); arr.reshape(2, 3, 4) |
| Flatten to 1D | arr.flatten() or arr.reshape(-1) | Converts a multi-dimensional array to a 1D array | arr = np.array([[1, 2], [3, 4]]) arr.flatten() |
| Transpose | arr.T or arr.transpose() | Swaps rows and columns | arr = np.array([[1, 2], [3, 4]]) arr.T |
| Reverse the reshape | .ravel .flatten | Get back to original | arr.ravel() arr.flatten() |
Combining Arrays
| Method | Syntax | Description | Example |
| Concatenation (Lists) | list1 + list2 | Creates a new list by joining elements from both lists | list1 = [1, 2] list2 = [3, 4] combined_list = list1 + list2 |
| Extend (Lists) | list1.extend(list2) | Appends elements from one list to another, modifying the original list | list1 = [1, 2] list2 = [3, 4] list1.extend(list2) |
| Concatenation (NumPy arrays) | np.concatenate((arr1, arr2), axis=0) | Concatenates arrays along a specified axis | arr1 = np.array([1, 2]) arr2 = np.array([3, 4]) combined_arr = np.concatenate((arr1, arr2)) |
| Stacking (NumPy arrays) | np.vstack((arr1, arr2)) or np.hstack((arr1, arr2)) | Stacks arrays vertically or horizontally | arr1 = np.array([1, 2]) arr2 = np.array([3, 4]) np.vstack((arr1, arr2)) V stack – ([[1, 2], [3, 4]]) np.hstack((arr1, arr2)) H stack – ([1, 2, 3, 4]) |
Testing for Conditions in NumPy Arrays
| Operation | Syntax | Description | Example |
| Element-wise comparison | array == value, array > value, etc. | Compares each element to a value, returning a boolean array | arr = np.array([1, 2, 3, 4]); result = arr > 2 |
| Logical operators | &, ` | ,~` | Performs logical AND, OR, and NOT operations on boolean arrays x[(x>5) & (x<10)] |
| where function | np.where(condition, x, y) | Returns elements from x where condition is True, otherwise from y | result = np.where(arr > 2, ‘greater than 2’, ‘less than or equal to 2’) |
| any function | np.any(condition) | Returns True if any element in the array satisfies the condition | result = np.any(arr > 3) |
| all function | np.all(condition) | Returns True if all elements in the array satisfy the condition | result = np.all(arr > 0) |
Broadcasting, vectorization, and arithmetic operations
| Concept | Description | Example |
| Broadcasting | Implicitly expands arrays to compatible shapes for arithmetic operations | a = np.array([1, 2, 3]) b = 2 result = a * b |
| Vectorization | Performs element-wise operations on arrays without explicit loops | a = np.array([1, 2, 3]) b = np.array([4, 5, 6]) result = a + b |
| Arithmetic operations | +, -, *, /, //, % | Basic arithmetic operations on arrays _and = np.logical_and(arr1,arr2) _or = np.logical_or(arr1,arr2) |
| Dot product | Matrix multiplication | np.dot(a,b) |
| Split to array | a = np.array([[1,3,6],[5,8,9],[5,5,6]]) np.hsplit(arr1, 3) np.vsplit(a,3) |
Masked array (allows nan values/missing or invalid data)
| Operation | Syntax | Description | Example |
| Creating a masked array | np.ma.masked_array(data, mask) | Creates a masked array from an array and a mask | data = np.array([1, 2, np.nan, 4, 5]) mask = np.isnan(data) masked_array = np.ma.masked_array(data, mask) |
| Accessing the data | masked_array.data | Returns the underlying data array | data = masked_array.data |
| Accessing the mask | masked_array.mask | Returns the mask array | mask = masked_array.mask |
| Replacing masked values | masked_array.filled(fill_value) | Replaces masked values with a specified value | filled_array = masked_array.filled(0) |
| Masked array operations | Arithmetic, comparison, etc. | Operations are performed on unmasked elements | result = masked_array * 2 |
| Masked array indexing | masked_array[mask] | Accesses unmasked elements | valid_values = masked_array[~masked_array.mask] |
import numpy as np
import numpy.ma as ma
data = np.array([1, 2, np.nan, 4, 5])
mask = np.isnan(data)
masked_array = ma.masked_array(data, mask)
print(masked_array)
# Output: masked_array(data=[1, 2, –, 4, 5],
mask=[False, False, True, False, False], fill_value=1e+20)
print(masked_array.mean())
# Output: 3.0
NumPy selecting arrays
| Operation | Syntax | Description | Example |
| Basic slicing | array[start:stop:step] [(rows) start:end:intervals/step, (columns) start:end:intervals/step] | Similar to lists, but supports multi-dimensional arrays | arr = np.array([[1, 2, 3], [4, 5, 6]]) sub_array = arr[1:, 1:] – array([[5, 6]]) for 3d array – arr[1,0,1] first row- arr[0,:] |
| Boolean indexing | array[boolean_mask] | Selects elements based on a boolean condition | mask = arr > 2 filtered_arr = arr[mask] – array([3, 4, 5, 6]) |
| Fancy indexing | array[index_array] | Selects elements based on an integer index array | indices = [0, 1] selected_elements = arr[indices] |
| Reverse | arr[::-1] np.flip(a) |
Other:
Type function to check data type – type(np.array([1,3]))
Data type – a.dtype
Size – a=np.arange(0,10).reshape(5,2); a.size
Dimension – a.ndim
Memory – a.nbytes
Memory – a.itemsize
Transpose – a.T or np.transpose(a)
Mean – a.mean()
Variance – a.var()
Standard deviation – a.std()
Sum for each column – a.sum(axis=0)
Cumulative sum – a.cumsum()
Max & Min – a.max(), a.min()
Matrix with semicolon – a=np.matrix(‘1,2;5,4’)
Product – a4 = np.prod(k2)
Find the index of the max & min value – np.argmax(a), np.argmin(a)
Trignometry – np.sin(a)
Exponent – np.exp(a)
Power – np.power(a,2)
Square root – np.sqrt(a)
Round – np.round(a), np.round(a,decimals=2)
Floor & ceil- np.floor(a), np.ceil(a)
Inverse of matrix – inverse_matrix = np.linalg.inv(matrix)
Sort – np.sort(a)
Add element in last – np.append(a,1)
Insert – b = np.random.randint(1,100,24).reshape(6,4), h= np.array([10,11,12,13]), np.insert(b,1, h, axis=0)
Unique – np.unique(e)
Where – a[np.where(a>50)]
Percentile – np.percentile(a,100) #max, np.percentile(a,0) # min, np.percentile(a,50) #median
Info – np.info(a)
Utility function – np.isin(a,items)
Delete – np.delete(a,1)
Set – np.union1d(m,n), np.intersect1d(m,n), np.setdiff1d(m,n), np.setxor1d(m,n), np.in1d (m,n)
