It really depends on what you're trying to do. First of all, you should never edit core WordPress files. Any changes you make will be lost if and when you upgrade, so if you want to change functionality, use a plug-in. If you want to change the way your site displays, change the theme.
In either case, you don't want to work on a live site ... things can go wrong, you might miss a semi-colon and break the site, or you might accidentally destroy something you've spent a long time building. That said, it's still pretty easy to get started with development.
First things first, you want to set up a test server on your local system and install WordPress there. There are several available, but here are a few good ones to look at first:
After you have your testing server set up, install the latest version of WordPress like you normally would. Now you have a standalone setup that you control completely and can test whatever you want without breaking your live site.
For development, you'll want to use some kind of IDE. Your choice of development environment is entirely up to you. Just make sure you comfortable using it. Some good (free) IDEs for PHP development are:
- Aptana Studio
- Microsoft Notepad - Not really an IDE, but if all you want to do is write a few lines of code and you don't care about intellisense, it's just fine.
- XCode - Mac only ... but I didn't want to be Windows exclusive :-)
As you develop, you'll want to track whatever changes you make so you can go back and "revert" any mistakes you've made. Most of us have done it several times ... you edit some code, think it works, and leave for the day. Then you come back and find out you broke something vital to the system ... but can't remember what you changed the day before to un-do the mistake! Using some kind of version control system will help you avoid this:
- Subversion - It's a bit older and can be harder to use, but tools like TortoiseSVN make it much easier to work with ... This is also the system used by WordPress to track core changes and submit plug-ins/themes to the repository.
This will get you started with developing for WordPress locally. You can build you plug-ins, themes, and other code and test it on the latest version of WordPress to make sure it works for you. Once it's done, tested, and ready for primetime you can upload it to your site or submit it to the WordPress repository.
Once you're ready to push your new code to the server, you have a few options. Plug-ins can be zipped locally and uploaded directly through the WordPress UI. Themes can, too. Alternatively, you can always FTP files directly to the server ... this allows you to make small changes in a large plug-in without needing to transfer the whole thing each time. Here are some good FTP tools:
There are several good tutorials available that discuss plug-in and theme development in detail. Some of the most important ones (i.e. the ones providing file templates and usable examples) are located in the Codex. A quick Google search will help you find even more: