The Golden Rule: Computers supplement, they don't replace.
Computers can also be addictive. If you start to find yourself getting into an addicitive cycle, regulate your behaviour as soon as you notice and you'll be fine.
A simple text editor can be much better than a pen and paper, it's much easier to make changes and I can type a lot faster than I can write. However, it's always good to have a pen and paper to hand, or to use the graphics tablet. A usable graphics tablet for general use is now quite affordable.
Application specific software (such as audio editors, sequencers, paint packages, mathematical software, programming environments) can also be of use, although this is a dicey area as these applications are almost always restrictive too. Think of them more as appliances than blank canvases. If they restrict you in any way, find another way, unless there is no other.
The internet, Wikipedia and Google in particular, make day to day research tasks trivial. You can fact find, routeplan, product search and many other tasks at a pinch that would otherwise have required making a trip or finding a knowledgeable person. It's easy for this to become a distraction though. Don't be too fascinated! And don't forget that making trips and conversing with people is vital as well.
Email and instant messaging are the backbone of a good computerised communication strategy. Mobile phones are great for brief conversations on the move, and texting is OK in an emergency (what a tedious data entry mechanism!). Webmail is great for checking your email on the go.
Bulletin boards and other online communities can be good, but they can also be big time drains. Find the right ones.
The web is full of things to read, videos to watch, games to play. It can all too easily get out of hand though.
I find that a good RSS reader, and ratings sites like del.icio.us, digg, stumbleupon etc. can really help make sense of things. Adding good bloggers and ratings sites to your feeds list can bring the best of the web right to you for little or no effort.
You can keep a lot of personal records nowadays. I use a removable USB drive for backups (NOT CDs or DVDs, passive backups are no good! Nothing substitutes for a live backup medium, especially for low volumes, and nowadays low can be anything up to a terabyte or two).
Storage costs are such that whenever my last pair of data disks (day-to-day use and backup) gets full I can replace them both with equivalents of twice the capacity. I never have to throw my data away, except to ease my mental load. I try to use simple file formats wherever I can, plain text, open formats, anything that's likely to be in use for a long time.
A digital camera is great for taking digital records of actual things.
A personal webspace and/or accounts on various websites can make it easy to publish things you've made, and easy to remotely access files. I use flickr, facebook, and some personal webspace, as well as a few other web accounts that I use less frequently such as YouTube and some subject-specific communities.
Managing the relationship between my accounts, my webspace and my local storage has been a bit of a headache: there is a lot of content on the web that I'd like to keep locally, but there is no simple method to do it. I think this is a general problem.
Internet privacy isn't nearly as important as people think. So long as you don't go smearing information that robots can gather (credit card details, passwords), or very high-value information, around on the internet, then you should be fine. There are a LOT of people clamouring for attention on the internet. You aren't special, and are very unlikely to receive much attention. That said, don't put anything on the internet that you wouldn't be happy to have broadcast on national television. If you stop to think about it though, that could include a lot more things than you might first think.