If you want to grow a garden that is free of chemicals, beneficial bugs are going to be a lifesaver for you. Without them, you'll end up with bugs all over your garden, killing plants, eating leaves, etc.
One of my favorites to use is the ladybug. They love aphids and can eat up a bunch of them. They'll also eat pollen as well, which means they may pollinate your plants. There's a good source of information about them here.
Ladybugs can be purchased at many feed stores, farm stores, etc. Here I've bought them at Coastal Farm & Ranch as well as Fred Meyer. They tend to come in a mesh bag or plastic cup. They tend to be in hibernation, but will wake up once they warm up. Don't be surprised if some are dead - that's normal. If a lot of them are dead, that's not, but don't be surprised if 10% or so are dead. That's why you get so many in the container.
Wait until it is almost dark and give your garden a good watering. The ladybugs are going to need a lot to drink once they get active. Then, just as it is getting dark, release your ladybugs. I tend to sprinkle them all over my garden so that I have good coverage. Ladybugs do not like to fly in the dark, which means they will stay in your garden. The longer you can get them to stay at the beginning, the more likely you are to keep some of them around.
When I came out the next morning, there were ladybugs everywhere. They were chasing bugs, eating pollen, and mating. I can only hope this means I'll find ladybug eggs later.
I also bought a little plastic house they had for ladybugs at the store. I am interested to see if this helps keep them around as well.