Group representation or linear representation

It is an action of the group on a vector space by means of linear maps. It can be seen like a functor if we think of a group as a category.
Given a group $G$, a representation is then a homomorphism $\rho :G\to GL(V)$ where $V$ is a vector space. If we fix a basis we can say it is a matrix representation.

A vector subspace $W\subset V$ is $G$-invariant if $\rho (g)v\in W$ for every $g\in G$ and $v\in W$. It can lead to a subrepresentation or a quotient representation.

A representation $\rho :G\to GL(V)$ is irreducible if it does not have non-trivial subrepresentations. Otherwise it is called reducible.

If the action of $G$ on $V-\{0\}$ is transitive then the representation is irreducible.

Since $\rho (G)$ is a subring of $End(V)$ then $V$ is a $\rho (G)$-module.

Related: spin representation.
Related: projective representation.
Important result: Schur's lemma.

Representations of $SU(2)$

There is exactly 1 irreducible representation of $SU(2)$ in $GL(n,\mathbb{C})$ for every $n\in \mathbb{N}$. See this video. SU(2)
According to @baez1994gauge page 174, physicists call to the representation of $SU(2)$ of dimension $n$ the spin-$j$ representation, with $j=(n-1)/2$.
To reconcile this with the mathematician language: $SU(2)$ is the spin group corresponding to $SO(3)$. The representations of $SU(2)$ of odd dimension ($j=0,1,2,\dots$) descend to a representation of $SO(3)$ (said in @baez1994gauge page 180) so they are not spin representations. But in the case of even dimension ($j=\frac{1}{2},\frac{3}{2},\frac{5}{2},\dots$), it doesn't descend, so it is indeed a proper spin representation.

Representations of $U(1)$

See @baez1994gauge page 171.
$U(1)$ is the unit circle in the complex plane, group operation is multiplication.
We have the representations defined by

for $n\in \mathbb{Z}$. These are irreducible because $\mathbb{C}$ (as a vector space) has no proper subspaces.

It turns out that every irreducible representation of $U(1)$ on a complex vector space is equivalent to ${\rho }_n$ for some $n$. To see it we use Schur's lemma and the fact that every complex 1-dimensional representation of $U(1)$ is equivalent to certain ${\rho }_n$.