PyTorch Day 02: PyTorch Tensors Basics

Table of Contents

1. Introduction
2. Understanding Tensors
   • 2.1. What is a Tensor?
   • 2.2. Tensor Attributes
     • 2.2.1. Shape
     • 2.2.2. Data Type (dtype)
     • 2.2.3. Device
3. Tensor Creation and Manipulation
   • 3.1. Creating Tensors
     • 3.1.1. Using torch.tensor
     • 3.1.2. Using torch.zeros
     • 3.1.3. Using torch.ones
     • 3.1.4. Using torch.arange
     • 3.1.5. Using torch.linspace
   • 3.2. Tensor Operations
     • 3.2.1. Arithmetic Operations
     • 3.2.2. Indexing and Slicing
     • 3.2.3. Reshaping Tensors
     • 3.2.4. Concatenation and Stacking
     • 3.2.5. Transposing Tensors
4. Practical Exercises
   • 4.1. Exercise 1: Exploring Tensor Attributes
   • 4.2. Exercise 2: Creating Various Tensors
   • 4.3. Exercise 3: Tensor Manipulation Operations
5. Solutions and Explanations
   • 5.1. Solutions to Practice Exercises
     • 5.1.1. Exercise 1: Exploring Tensor Attributes
     • 5.1.2. Exercise 2: Creating Various Tensors
     • 5.1.3. Exercise 3: Tensor Manipulation Operations
6. Summary
7. Additional Resources

---

1. Introduction

Tensors are the fundamental building blocks in PyTorch and deep learning. They are multi-dimensional arrays that enable efficient computation, especially when leveraging GPUs for acceleration. Understanding tensors' structure, attributes, and manipulation techniques is essential for developing and training deep learning models effectively. Today's guide will equip you with the knowledge to create and manipulate tensors confidently, setting the stage for more advanced topics in the days to come.

---

2. Understanding Tensors

2.1. What is a Tensor?

A tensor is a generalization of vectors and matrices to potentially higher dimensions. In PyTorch, tensors are the primary data structures used to store data and model parameters. They are similar to NumPy's ndarray but come with additional capabilities, such as GPU acceleration and automatic differentiation.

Key Characteristics of Tensors:

• Multi-Dimensional: Tensors can have any number of dimensions, making them versatile for various data representations.
• Efficient Computation: Optimized for high-performance mathematical operations, especially on GPUs.
• Gradients: Supports automatic differentiation, crucial for training neural networks.

Common Tensor Ranks:

• 0-D Tensor (Scalar): A single value.
• 1-D Tensor (Vector): A one-dimensional array.
• 2-D Tensor (Matrix): A two-dimensional array.
• 3-D and Higher: Used for more complex data like images (3-D), videos (4-D), etc.

Visualization:

• 0-D: tensor(5)
• 1-D: tensor([1, 2, 3])
• 2-D: tensor([[1, 2], [3, 4]])
• 3-D: tensor([[[1], [2]], [[3], [4]]])

2.2. Tensor Attributes

Understanding a tensor's attributes is crucial for effective manipulation and ensuring compatibility during operations.

2.2.1. Shape

• Definition: Represents the size of the tensor in each dimension.
• Access: tensor.shape or tensor.size()

Example:

import torch

tensor = torch.randn(3, 4, 5)
print("Tensor Shape:", tensor.shape)  # Output: torch.Size([3, 4, 5])

2.2.2. Data Type (dtype)

• Definition: Indicates the type of data stored in the tensor (e.g., float, int).
• Access: tensor.dtype

Common Data Types:

• torch.float32 (torch.float)
• torch.float64 (torch.double)
• torch.int32 (torch.int)
• torch.int64 (torch.long)
• torch.bool

Example:

tensor = torch.randn(3, 4)
print("Tensor Data Type:", tensor.dtype)  # Output: torch.float32

2.2.3. Device

• Definition: Specifies where the tensor is stored and processed (CPU or GPU).
• Access: tensor.device

Devices:

• cpu
• cuda:0, cuda:1, etc., representing different GPUs.

Example:

tensor = torch.randn(2, 2)
print("Tensor Device:", tensor.device)  # Output: cpu

# Moving tensor to GPU (if available)
if torch.cuda.is_available():
    tensor_gpu = tensor.to('cuda')
    print("Tensor Device after moving to GPU:", tensor_gpu.device)  # Output: cuda:0

---

3. Tensor Creation and Manipulation

Creating tensors in various ways allows flexibility in initializing data for different tasks. PyTorch provides a plethora of functions to create tensors with desired properties.

3.1. Creating Tensors

3.1.1. Using torch.tensor

Creates a tensor from data (e.g., lists, tuples).

Example:

import torch

# From a list
data = [[1, 2], [3, 4]]
tensor = torch.tensor(data)
print("Tensor from list:\n", tensor)
# Output:
# Tensor from list:
# tensor([[1, 2],
#         [3, 4]])

Specifying Data Type:

tensor = torch.tensor(data, dtype=torch.float32)
print("Tensor with dtype float32:\n", tensor)
# Output:
# tensor([[1., 2.],
#         [3., 4.]])

3.1.2. Using torch.zeros

Creates a tensor filled with zeros.

Example:

tensor = torch.zeros(3, 4)
print("Tensor filled with zeros:\n", tensor)
# Output:
# Tensor filled with zeros:
# tensor([[0., 0., 0., 0.],
#         [0., 0., 0., 0.],
#         [0., 0., 0., 0.]])

3.1.3. Using torch.ones

Creates a tensor filled with ones.

Example:

tensor = torch.ones(2, 3)
print("Tensor filled with ones:\n", tensor)
# Output:
# Tensor filled with ones:
# tensor([[1., 1., 1.],
#         [1., 1., 1.]])

3.1.4. Using torch.arange

Creates a 1-D tensor with values from a start to an end with a specified step.

Example:

tensor = torch.arange(start=0, end=10, step=2)
print("Tensor created with arange:\n", tensor)
# Output:
# Tensor created with arange:
# tensor([0, 2, 4, 6, 8])

3.1.5. Using torch.linspace

Creates a 1-D tensor with values linearly spaced between start and end.

Example:

tensor = torch.linspace(start=0, end=1, steps=5)
print("Tensor created with linspace:\n", tensor)
# Output:
# Tensor created with linspace:
# tensor([0.0000, 0.2500, 0.5000, 0.7500, 1.0000])

3.2. Tensor Operations

Once tensors are created, various operations can be performed to manipulate and analyze data.

3.2.1. Arithmetic Operations

PyTorch supports element-wise arithmetic operations between tensors.

Example:

import torch

tensor_a = torch.tensor([1, 2, 3])
tensor_b = torch.tensor([4, 5, 6])

# Addition
sum_tensor = tensor_a + tensor_b
print("Sum:", sum_tensor)  # Output: tensor([5, 7, 9])

# Multiplication
prod_tensor = tensor_a * tensor_b
print("Product:", prod_tensor)  # Output: tensor([ 4, 10, 18])

# Division
div_tensor = tensor_b / tensor_a
print("Division:", div_tensor)  # Output: tensor([4.0000, 2.5000, 2.0000])

Scalar Operations:

# Addition with scalar
tensor = torch.tensor([1, 2, 3])
tensor = tensor + 5
print("After adding 5:", tensor)  # Output: tensor([6, 7, 8])

# Multiplication with scalar
tensor = tensor * 2
print("After multiplying by 2:", tensor)  # Output: tensor([12, 14, 16])

3.2.2. Indexing and Slicing

Accessing specific elements or subsets of a tensor is fundamental for data manipulation.

Example:

import torch

tensor = torch.arange(1, 17).reshape(4, 4)
print("Original Tensor:\n", tensor)
# Output:
# Original Tensor:
# tensor([[ 1,  2,  3,  4],
#         [ 5,  6,  7,  8],
#         [ 9, 10, 11, 12],
#         [13, 14, 15, 16]])

# Accessing a single element
element = tensor[0, 1]
print("Element at (0,1):", element)  # Output: tensor(2)

# Slicing rows
rows = tensor[1:3, :]
print("Sliced Rows (1:3):\n", rows)
# Output:
# Sliced Rows (1:3):
# tensor([[ 5,  6,  7,  8],
#         [ 9, 10, 11, 12]])

# Slicing columns
columns = tensor[:, 2:]
print("Sliced Columns (2:):\n", columns)
# Output:
# Sliced Columns (2:):
# tensor([[ 3,  4],
#         [ 7,  8],
#         [11, 12],
#         [15, 16]])

# Using negative indices
last_element = tensor[-1, -1]
print("Last Element:", last_element)  # Output: tensor(16)

3.2.3. Reshaping Tensors

Reshaping changes the tensor's dimensions without altering its data.

Using reshape:

import torch

tensor = torch.arange(1, 13)
print("Original Tensor Shape:", tensor.shape)  # Output: torch.Size([12])

reshaped = tensor.reshape(3, 4)
print("Reshaped Tensor Shape:", reshaped.shape)
# Output:
# Reshaped Tensor Shape: torch.Size([3, 4])

Using view:

reshaped_view = tensor.view(3, 4)
print("Reshaped with view Shape:", reshaped_view.shape)
# Output:
# Reshaped with view Shape: torch.Size([3, 4])

Note: The tensor must be contiguous in memory when using view. If not, use reshape or .contiguous() before view.

3.2.4. Concatenation and Stacking

Combining tensors is often necessary for data aggregation and model input preparation.

Concatenation (torch.cat):

• Description: Joins tensors along an existing dimension.
• Requirements: Tensors must have the same shape except in the concatenating dimension.

Example:

import torch

tensor_a = torch.randn(2, 3)
tensor_b = torch.randn(2, 3)

# Concatenate along the first dimension (rows)
concat_dim0 = torch.cat((tensor_a, tensor_b), dim=0)
print("Concatenated along dim=0:", concat_dim0.shape)  # Output: torch.Size([4, 3])

# Concatenate along the second dimension (columns)
concat_dim1 = torch.cat((tensor_a, tensor_b), dim=1)
print("Concatenated along dim=1:", concat_dim1.shape)  # Output: torch.Size([2, 6])

Stacking (torch.stack):

• Description: Joins tensors along a new dimension.
• Requirements: All tensors must have the same shape.

Example:

import torch

tensor_a = torch.randn(2, 3)
tensor_b = torch.randn(2, 3)

# Stack along a new first dimension
stack_dim0 = torch.stack((tensor_a, tensor_b), dim=0)
print("Stacked along new dim=0:", stack_dim0.shape)  # Output: torch.Size([2, 2, 3])

# Stack along a new second dimension
stack_dim1 = torch.stack((tensor_a, tensor_b), dim=1)
print("Stacked along new dim=1:", stack_dim1.shape)  # Output: torch.Size([2, 2, 3])

3.2.5. Transposing Tensors

Transposing swaps two dimensions of a tensor, which is essential for aligning data for specific operations.

Using transpose:

import torch

tensor = torch.randn(2, 3)
print("Original Tensor Shape:", tensor.shape)  # Output: torch.Size([2, 3])

# Transpose dimensions 0 and 1
transposed = tensor.transpose(0, 1)
print("Transposed Tensor Shape:", transposed.shape)  # Output: torch.Size([3, 2])

Using permute:

import torch

tensor = torch.randn(2, 3, 4)
print("Original Tensor Shape:", tensor.shape)  # Output: torch.Size([2, 3, 4])

# Permute dimensions to (3, 4, 2)
permuted = tensor.permute(1, 2, 0)
print("Permuted Tensor Shape:", permuted.shape)  # Output: torch.Size([3, 4, 2])

---

4. Practical Exercises

Engaging with hands-on exercises reinforces your understanding and ensures you can apply tensor operations effectively.

4.1. Exercise 1: Exploring Tensor Attributes

Task:

1. Create a 3-D tensor with shape (4, 3, 2).
2. Print its shape, data type, and device.
3. Change the data type to torch.float64 and move it to the GPU (if available).
4. Verify the changes by printing the updated attributes.

4.2. Exercise 2: Creating Various Tensors

Task:

1. Create the following tensors:
   • A tensor of zeros with shape (5, 5).
   • A tensor of ones with shape (3, 4).
   • A tensor with values from 0 to 9 using torch.arange.
   • A tensor with 50 linearly spaced points between 0 and 1 using torch.linspace.
2. Print each tensor and its attributes.

4.3. Exercise 3: Tensor Manipulation Operations

Task:

1. Create two 2-D tensors:
   • tensor_a with shape (2, 3) containing random values.
   • tensor_b with shape (2, 3) containing random values.
2. Perform the following operations:
   • Element-wise addition.
   • Element-wise multiplication.
   • Matrix multiplication (torch.matmul).
3. Slice tensor_a to extract the first two elements of each row.
4. Reshape tensor_b to (3, 2).
5. Concatenate tensor_a and tensor_b along the first dimension.
6. Stack tensor_a and tensor_b along a new dimension.
7. Transpose the stacked tensor.
8. Print the results of each operation.

---

5. Solutions and Explanations

5.1. Solutions to Practice Exercises

5.1.1. Exercise 1: Exploring Tensor Attributes

Solution:

import torch

# Step 1: Create a 3-D tensor with shape (4, 3, 2)
tensor = torch.randn(4, 3, 2)
print("Original Tensor:\n", tensor)
print("Shape:", tensor.shape)           # torch.Size([4, 3, 2])
print("Data Type:", tensor.dtype)       # torch.float32
print("Device:", tensor.device)         # cpu

# Step 2: Change data type to torch.float64 and move to GPU if available
if torch.cuda.is_available():
    tensor = tensor.to(dtype=torch.float64, device='cuda')
else:
    tensor = tensor.to(dtype=torch.float64)

print("\nUpdated Tensor:\n", tensor)
print("Shape:", tensor.shape)           # torch.Size([4, 3, 2])
print("Data Type:", tensor.dtype)       # torch.float64
print("Device:", tensor.device)         # cuda:0 or cpu

Explanation:

1. Tensor Creation: Initializes a 3-D tensor with random values and shape (4, 3, 2).
2. Attribute Printing: Displays the tensor's shape, data type (torch.float32 by default), and device (cpu).
3. Data Type and Device Change: Uses the .to() method to change the data type to torch.float64 and moves the tensor to the GPU (cuda) if available.
4. Verification: Prints the updated tensor's attributes to confirm the changes.

Sample Output:

Original Tensor:
 tensor([[[ 0.1234, -0.5678],
         [ 1.2345, -1.3456],
         [ 0.6789, -0.7890]],

        [[-0.2345,  1.4567],
         [ 0.3456, -1.5678],
         [ 1.7890, -0.8901]],

        [[-1.1234,  0.2345],
         [ 0.5678, -1.6789],
         [ 1.8901, -0.3456]],

        [[ 0.4567, -1.7890],
         [ 1.2345, -0.4567],
         [ 0.6789, -1.8901]]])
Shape: torch.Size([4, 3, 2])
Data Type: torch.float32
Device: cpu

Updated Tensor:
 tensor([[[ 0.1234, -0.5678],
         [ 1.2345, -1.3456],
         [ 0.6789, -0.7890]],

        [[-0.2345,  1.4567],
         [ 0.3456, -1.5678],
         [ 1.7890, -0.8901]],

        [[-1.1234,  0.2345],
         [ 0.5678, -1.6789],
         [ 1.8901, -0.3456]],

        [[ 0.4567, -1.7890],
         [ 1.2345, -0.4567],
         [ 0.6789, -1.8901]]], dtype=torch.float64, device='cuda:0')
Shape: torch.Size([4, 3, 2])
Data Type: torch.float64
Device: cuda:0

5.1.2. Exercise 2: Creating Various Tensors

Solution:

import torch

# 1. Tensor of zeros with shape (5, 5)
zeros_tensor = torch.zeros(5, 5)
print("Tensor of Zeros:\n", zeros_tensor)
print("Shape:", zeros_tensor.shape)
print("Data Type:", zeros_tensor.dtype)
print("Device:", zeros_tensor.device)

# 2. Tensor of ones with shape (3, 4)
ones_tensor = torch.ones(3, 4)
print("\nTensor of Ones:\n", ones_tensor)
print("Shape:", ones_tensor.shape)
print("Data Type:", ones_tensor.dtype)
print("Device:", ones_tensor.device)

# 3. Tensor with values from 0 to 9 using torch.arange
arange_tensor = torch.arange(0, 10)
print("\nTensor with torch.arange:\n", arange_tensor)
print("Shape:", arange_tensor.shape)
print("Data Type:", arange_tensor.dtype)
print("Device:", arange_tensor.device)

# 4. Tensor with 50 linearly spaced points between 0 and 1 using torch.linspace
linspace_tensor = torch.linspace(0, 1, steps=50)
print("\nTensor with torch.linspace:\n", linspace_tensor)
print("Shape:", linspace_tensor.shape)
print("Data Type:", linspace_tensor.dtype)
print("Device:", linspace_tensor.device)

Explanation:

1. Tensor of Zeros: Creates a (5, 5) tensor filled with zeros.
2. Tensor of Ones: Creates a (3, 4) tensor filled with ones.
3. Tensor with torch.arange: Generates a 1-D tensor with values from 0 to 9.
4. Tensor with torch.linspace: Generates a 1-D tensor with 50 evenly spaced values between 0 and 1.
5. Attribute Printing: For each tensor, prints its shape, data type, and device to verify creation.

Sample Output:

Tensor of Zeros:
 tensor([[0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0.]])
Shape: torch.Size([5, 5])
Data Type: torch.float32
Device: cpu

Tensor of Ones:
 tensor([[1., 1., 1., 1.],
        [1., 1., 1., 1.],
        [1., 1., 1., 1.]])
Shape: torch.Size([3, 4])
Data Type: torch.float32
Device: cpu

Tensor with torch.arange:
 tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
Shape: torch.Size([10])
Data Type: torch.int64
Device: cpu

Tensor with torch.linspace:
 tensor([0.0000, 0.0204, 0.0408, 0.0612, 0.0816, 0.1020, 0.1224, 0.1429,
        0.1633, 0.1837, 0.2041, 0.2245, 0.2449, 0.2653, 0.2857, 0.3061,
        0.3265, 0.3469, 0.3673, 0.3878, 0.4082, 0.4286, 0.4490, 0.4694,
        0.4898, 0.5102, 0.5306, 0.5510, 0.5714, 0.5918, 0.6122, 0.6327,
        0.6531, 0.6735, 0.6939, 0.7143, 0.7347, 0.7551, 0.7755, 0.7959,
        0.8163, 0.8367, 0.8571, 0.8776, 0.8980, 0.9184, 0.9388, 0.9592,
        0.9796, 1.0000])
Shape: torch.Size([50])
Data Type: torch.float32
Device: cpu

5.1.3. Exercise 3: Tensor Manipulation Operations

Solution:

import torch

# Step 1: Create two 2-D tensors
tensor_a = torch.randn(2, 3)
tensor_b = torch.randn(2, 3)
print("Tensor A:\n", tensor_a)
print("Shape of Tensor A:", tensor_a.shape)  # torch.Size([2, 3])

print("\nTensor B:\n", tensor_b)
print("Shape of Tensor B:", tensor_b.shape)  # torch.Size([2, 3])

# Step 2.1: Element-wise addition
addition = tensor_a + tensor_b
print("\nElement-wise Addition:\n", addition)

# Step 2.2: Element-wise multiplication
multiplication = tensor_a * tensor_b
print("\nElement-wise Multiplication:\n", multiplication)

# Step 2.3: Matrix multiplication
# For matrix multiplication, tensors must be compatible. Here, (2,3) @ (3,2) = (2,2)
tensor_b_mat = tensor_b.T  # Transpose tensor_b to make it (3,2)
matrix_multiplication = torch.matmul(tensor_a, tensor_b_mat)
print("\nMatrix Multiplication (A @ B^T):\n", matrix_multiplication)

# Step 3: Slice tensor_a to extract the first two elements of each row
sliced = tensor_a[:, :2]
print("\nSliced Tensor A (first two elements of each row):\n", sliced)
print("Shape of Sliced Tensor:", sliced.shape)  # torch.Size([2, 2])

# Step 4: Reshape tensor_b to (3, 2)
reshaped_b = tensor_b.reshape(3, 2)
print("\nReshaped Tensor B to (3, 2):\n", reshaped_b)
print("Shape of Reshaped Tensor B:", reshaped_b.shape)  # torch.Size([3, 2])

# Step 5: Concatenate tensor_a and reshaped_b along the first dimension
# To concatenate, shapes must match except in the concatenating dimension.
# tensor_a: (2,3), reshaped_b: (3,2) --> Not directly compatible. Adjust reshaped_b to (3,3) by padding or similar.
# Alternatively, stack tensors along a new dimension if shapes are incompatible for concatenation.
# Here, let's stack along a new dimension instead.
stacked = torch.stack((tensor_a, tensor_b), dim=0)
print("\nStacked Tensor A and B along new dimension 0:\n", stacked)
print("Shape of Stacked Tensor:", stacked.shape)  # torch.Size([2, 2, 3])

# Step 6: Transpose the stacked tensor (swap dimensions 1 and 2)
transposed_stacked = stacked.transpose(1, 2)
print("\nTransposed Stacked Tensor (swap dim 1 and 2):\n", transposed_stacked)
print("Shape of Transposed Stacked Tensor:", transposed_stacked.shape)  # torch.Size([2, 3, 2])

Explanation:

1. Tensor Creation: Initializes two random (2, 3) tensors, tensor_a and tensor_b.
2. Arithmetic Operations:
   • Element-wise Addition: Adds corresponding elements of tensor_a and tensor_b.
   • Element-wise Multiplication: Multiplies corresponding elements of tensor_a and tensor_b.
   • Matrix Multiplication: Performs matrix multiplication between tensor_a and the transpose of tensor_b to ensure dimensional compatibility, resulting in a (2, 2) tensor.
3. Slicing: Extracts the first two elements from each row of tensor_a, resulting in a (2, 2) tensor.
4. Reshaping: Changes the shape of tensor_b from (2, 3) to (3, 2).
5. Concatenation and Stacking:
   • Concatenation: Attempted but found incompatible due to shape mismatch.
   • Stacking: Instead, stacks tensor_a and tensor_b along a new dimension, resulting in a (2, 2, 3) tensor.
6. Transposing Stacked Tensor: Swaps the second and third dimensions of the stacked tensor, resulting in a (2, 3, 2) tensor.
7. Attribute Printing: After each operation, prints the resulting tensor and its shape for verification.

Sample Output:

Tensor A:
 tensor([[ 0.1234, -1.2345,  0.5678],
        [ 1.2345, -0.5678,  1.3456]])
Shape of Tensor A: torch.Size([2, 3])

Tensor B:
 tensor([[ 0.6789, -1.8901,  0.4567],
        [ 1.8901, -0.3456,  1.6789]])
Shape of Tensor B: torch.Size([2, 3])

Element-wise Addition:
 tensor([[ 0.8023, -3.1246,  2.1245],
        [ 3.1246, -0.9134,  3.0245]])

Element-wise Multiplication:
 tensor([[ 0.0841,  2.1974,  0.2578],
        [ 2.3456,  0.1944,  2.2690]])

Matrix Multiplication (A @ B^T):
 tensor([[ 0.1234*0.6789 + (-1.2345)*1.8901 + 0.5678*1.8901,
          0.1234*(-1.8901) + (-1.2345)*(-0.3456) + 0.5678*1.6789],
        [1.2345*0.6789 + (-0.5678)*1.8901 + 1.3456*1.8901,
         1.2345*(-1.8901) + (-0.5678)*(-0.3456) + 1.3456*1.6789]])
# Actual numerical values will vary based on random initialization.

Sliced Tensor A (first two elements of each row):
 tensor([[ 0.1234, -1.2345],
        [ 1.2345, -0.5678]])
Shape of Sliced Tensor: torch.Size([2, 2])

Reshaped Tensor B to (3, 2):
 tensor([[ 0.6789, -1.8901],
        [ 0.4567,  1.8901],
        [-0.3456,  1.6789]])
Shape of Reshaped Tensor B: torch.Size([3, 2])

Stacked Tensor A and B along new dimension 0:
 tensor([[[ 0.1234, -1.2345,  0.5678],
         [ 1.2345, -0.5678,  1.3456]],

        [[ 0.6789, -1.8901,  0.4567],
         [ 1.8901, -0.3456,  1.6789]]])
Shape of Stacked Tensor: torch.Size([2, 2, 3])

Transposed Stacked Tensor (swap dim 1 and 2):
 tensor([[[ 0.1234,  1.2345],
         [-1.2345, -0.5678],
         [ 0.5678,  1.3456]],

        [[ 0.6789,  1.8901],
         [-1.8901, -0.3456],
         [ 0.4567,  1.6789]]])
Shape of Transposed Stacked Tensor: torch.Size([2, 3, 2])

Note: The numerical values will vary each time you run the script due to the use of random tensor initialization.

---

6. Summary

Today, you've delved into the basics of PyTorch tensors, exploring their structure, attributes, and various creation and manipulation techniques. Here's a concise recap of what you've accomplished:

• Understanding Tensors:

  • Grasped the definition and significance of tensors in deep learning.
  • Learned about tensor ranks (dimensions) and how they represent different data structures.
  • Explored tensor attributes: shape, data type (dtype), and device placement (CPU/GPU).

• Tensor Creation:

  • Created tensors using diverse methods like torch.tensor, torch.zeros, torch.ones, torch.arange, and torch.linspace.
  • Understood how to specify data types and devices during tensor creation.

• Tensor Manipulation:

  • Performed arithmetic operations (addition, multiplication, division) both element-wise and via matrix multiplication.
  • Executed indexing and slicing to access and modify specific tensor elements.
  • Reshaped tensors using reshape and view, ensuring dimensional compatibility.
  • Concatenated and stacked tensors to combine data along existing or new dimensions.
  • Transposed tensors using transpose and permute to rearrange dimensions for various computational needs.

• Practical Application:

  • Applied tensor operations through hands-on exercises, reinforcing theoretical knowledge with practical skills.

Mastering these tensor operations is pivotal as tensors form the backbone of all computations in deep learning models. The ability to create, manipulate, and efficiently utilize tensors will empower you to build and train complex neural networks effectively.

---

7. Additional Resources

To further enhance your understanding and proficiency with PyTorch tensors and deep learning fundamentals, consider exploring the following resources:

• Official PyTorch Documentation:

  • PyTorch Tensors
  • Tensor Attributes
  • Tensor Creation
  • Tensor Manipulation

• PyTorch Tutorials:

  • PyTorch Tutorial: Tensors
  • Deep Learning with PyTorch: A 60 Minute Blitz

• Books:

  • Deep Learning with PyTorch by Eli Stevens, Luca Antiga, and Thomas Viehmann.
  • Programming PyTorch for Deep Learning by Ian Pointer.

• Online Courses:

  • Coursera: Introduction to Deep Learning with PyTorch
  • Udacity: Intro to Machine Learning with PyTorch

• Community Forums and Support:

  • PyTorch Forums
  • Stack Overflow PyTorch Tag
  • GitHub PyTorch Issues

• Blogs and Articles:

  • Towards Data Science PyTorch Articles
  • PyTorch Official Blog

• YouTube Channels:

  • PyTorch
  • Sentdex – Offers practical PyTorch tutorials.

Tips for Continued Learning:

• Hands-On Practice: Regularly implement tensor operations and experiment with different scenarios to deepen your understanding.
• Engage with the Community: Participate in forums, ask questions, and collaborate on projects to gain diverse insights.
• Build Projects: Apply your tensor manipulation skills in real-world projects, such as image classification, natural language processing, or time-series analysis.
• Stay Updated: Follow PyTorch's official channels and repositories to keep abreast of the latest updates, features, and best practices.

By leveraging these resources and maintaining a consistent practice routine, you'll develop a robust mastery of PyTorch tensors, paving the way for advanced deep learning endeavors.

---

Happy Learning and Coding!