You should have done that four years ago. How you do it depends on your motherboard, so try to find your manual. Some come with software where you can set the multiplier directly in windows, others you have to go into bios.
As a baseline you want to try to set the multiplier at 45. If it's stable you go up, if you crash you go down until you find some number that works. A 2500k should be able to do over 40 with no additional tweaking. To get the max performance, you need to mess with voltages and such, but as a general thing you can get a lot better performance simply by editing the multiplier and leaving everything else in place.