In a normal crystal, the atoms are arranged in a repeating pattern, that repeats exactly throughout the whole crystal in all three dimensions (we call this periodicity). In order to create such repeating arrangements, the atoms are often arranged on the corners of triangles, squares or hexagons (all shapes that can tessellate with just themselves to completely cover an area).
In a quasi-crystal, the atoms are arranged in an ordered way, but this pattern can not repeat itself through the whole crystal, as the atoms are arranged on pentagons, which can’t tessellate.
When quasi-crystals were discovered, they turned the whole world of cystallography (the study of crystal structures) upside down, and Dan Shechtman (their discoverer who won the Nobel prize in Chemistry in 2011) was rejected by certain members of the Chemistry community for this discovery. The definition of what a crystal is was then changed to include quasi-crystals too.
To understand a quasicrystal, you must first understand what a crystal is. A crystal is an ordered, PERIODIC structure. So there is some unit of structure that repeats in a certain direction. One way to think about this is that you looked at a crystal structure, and moved it some distance in a certain direction, you could make it look exactly the same as it was before, this called translational symmetry.
A quasicrystal has an ordered structure, but it is not periodic. This means that if you look at it from a certain place, there is no way you can shift it around so that it looks the same again, so they lack translational symmetry.
Another nice thing is that normal crystals can only show rotational symmetry of certain orders (2,3,4 and 6 fold rotational symmetries in fact, this wikipedia article does a good job of describing rotational symmetry: http://en.wikipedia.org/wiki/Rotational_symmetry). Quasicrystals, however, can show rotational symmetry in orders other than these, for example, the famous 5-fold symmetry.