Our 'advent' calendar each year is just a a paper chain. We write down things to do on each of the chains. Here are some examples.
In addition to the usual paper chains and paper stars, there are other simple and frugal decorations you can make using paper scraps.
Use old pages from books, magazines, song books, piano music, or print them from online books if you can't tear up an old book that has outlived its value (I used books that were already missing pages) to make these paper hearts. We're planning a tree garland with ours. Instructions here.
Remember these from Kindergarten? These are easy to make, and so pretty as well as frugal when you use scraps of wrapping paper.