But the goat chosen by lot as the scapegoat shall be presented alive before the LORD to be used for making atonement by sending it into the desert as a scapegoat.  Leviticus 16:10

For us today, a scapegoat is someone who takes the blame for a problem or situation which he did not cause.

In biblical terms, it meant the same thing...literally.

One time a year, on the Day of Atonement, a priest was to pick two goats.  One would be sacrificed for the sins of the people and the other would be the scapegoat (in Hebrew aza'zel, a word that only appears in Leviticus 16).

The priests would lay their hands on the goat (Leviticus 16:20), confessing all the wickedness and rebellion of the Israelites, and place those sins ceremonially on the goat.  The goat would then be set free into the wilderness, far away from the people, carrying all those sins with it.

The Day of Atonement was a day to show that the people were forgiven of their sins (one goat had taken on the penalty of their sins by dying) and their sins were removed from them and out of God's sight (the scapegoat who wandered far away).  It was a day of Sabbath rest, meaning the people were to do nothing but rest and relax.

It's a perfect picture of what Jesus did for us.  He died for our sins, then removed them from God's sight.  He became our sacrifice and our scapegoat.  Our sins were all placed on him.  For us, now, all we do is rest.  There is nothing more we need to do.  God took care of all of it for us.