# Tensors & Devices

Use this page to quickly reference how to create and inspect tensors, change their shape or dtype, convert between NumPy and PyTorch, and move tensors and models between devices.

## Tensor Basics

### Creating Tensors

Use `torch.tensor` to construct a tensor from existing Python data.

```python
x = torch.tensor([1.0, 2.0, 3.0])

x = torch.tensor([[1, 2], 
                  [3, 4]], dtype=torch.float32)
```

Common constructors create tensors from a requested shape or numerical sequence.

| Constructor | Returns |
| --- | --- |
| `torch.zeros(shape)` | Tensor of zeros |
| `torch.ones(shape)` | Tensor of ones |
| `torch.arange(start, end)` | Tensor of integers from `start` to `end - 1` |
| `torch.randn(shape)` | Tensor of random numbers from a standard normal distribution |


### Inspecting Tensors

The most useful tensor properties are available directly on the tensor.

| Property | Returns |
| --- | --- |
| `x.shape` | Size of each dimension |
| `x.ndim` | Number of dimensions |
| `x.numel()` | Total number of elements |
| `x.dtype` | Data type |
| `x.device` | Device containing the tensor |


### Dtypes

The course mainly uses three tensor dtypes.

| Typical use | dtype |
| --- | --- |
| Features, activations, parameters | `torch.float32` |
| Multiclass target indices | `torch.int64` |
| Boolean masks | `torch.bool` |

Convert a tensor with `.to(...)`:

```python
x = x.to(torch.float32)
targets = targets.to(torch.int64)
```

Convenience methods are also available:

```python
x = x.float()
targets = targets.long()
```

:::{warning}
For multiclass classification, `nn.CrossEntropyLoss` expects targets to be integer class indices.
:::



### NumPy and PyTorch

Convert a NumPy array to a tensor with `torch.from_numpy`.

```python
import numpy as np

array = np.array([1.0, 2.0, 3.0], dtype=np.float32)

x = torch.from_numpy(array)
```

Convert a CPU tensor to a NumPy array with `.numpy()`.

```python
array = x.numpy()
```

`torch.from_numpy` shares memory with the NumPy array, so changing one can change the other.

For a tensor produced during model computation, a useful conversion pattern is:

```python
array = tensor.detach().cpu().numpy()
```



## Selecting and Reshaping

### Indexing and Slicing

PyTorch tensors use NumPy-like indexing.

```python
x[0]        # first element along dimension 0
x[:, 0]     # first column
x[2:5]      # elements 2, 3, and 4
x[..., 0]   # index 0 along the final dimension
```

For a batch of images:

```python
images.shape  # → (N, C, H, W)

first_image = images[0]  # → (C, H, W)

first_channel = images[:, 0]  # → (N, H, W)
```

Boolean tensors can be used as masks.

```python
mask = targets == 3
selected = features[mask]
```



### `reshape`

`reshape` changes the dimensions of a tensor without changing its number of elements.

```python
x = torch.arange(12)
y = x.reshape(3, 4)

# y.shape → (3, 4)
```

Use `-1` to let PyTorch infer one dimension.

```python
x = torch.randn(32, 1, 28, 28)

flattened = x.reshape(x.shape[0], -1)  

# flattened.shape → (32, 784)
```

:::{warning}
The new shape must contain the same total number of elements as the original tensor.
:::



### `flatten`

`flatten` combines several dimensions into one.

```python
x = torch.randn(2, 3, 4)

y = x.flatten()

# y.shape → (24, )
```

For batched data, preserve the batch dimension with `start_dim=1`.

```python
images = torch.randn(64, 1, 28, 28)

features = images.flatten(start_dim=1)

# features.shape → (64, 784)
```

:::{warning}
Calling `flatten()` without `start_dim=1` also flattens the batch dimension.
:::



### `unsqueeze` and `squeeze`

`unsqueeze` inserts a dimension of size one.

```python
image = torch.randn(3, 28, 28)

batch = image.unsqueeze(0)

# batch.shape → (1, 3, 28, 28)
```

`squeeze` removes a dimension of size one.

```python
image = batch.squeeze(0)

# image.shape → (3, 28, 28)
```

Specify the dimension when its position is known.

```python
x = x.squeeze(1)
```

:::{warning}
Calling `squeeze()` without a dimension removes every dimension whose size is one, which can unintentionally remove a batch dimension when the batch contains one example.
:::



## Devices

### Selecting a Device

Use an available accelerator when possible and fall back to the CPU.

```python
if torch.cuda.is_available():
    device = torch.device("cuda")
elif torch.backends.mps.is_available():
    device = torch.device("mps")
else:
    device = torch.device("cpu")
```



### Moving Tensors and Models

Move the model and the tensors involved in the same computation to the selected device.

```python
model = model.to(device)
inputs = inputs.to(device)
targets = targets.to(device)
```

:::{warning}
Moving the model does not automatically move the input batch.
:::



### Inspecting Device Placement

Inspect a tensor directly:

```python
inputs.device
```

For a model, inspect one of its parameters:

```python
next(model.parameters()).device
```

A useful consistency check is:

```python
assert inputs.device == next(model.parameters()).device
```