Pretty much. Do this:
Go to 'Start' button.
Click 'Run'
Enter 'cmd' without the quotes
Click 'Enter'
Type in 'ipconfig' without the quotes and hit enter
It'll show you your IP address. That's the Ip address that you want to enter.
However, the problem comes about if your router is 'DHCP' enabled. If there are other computers connected to the router, your router will assign IP addresses to all of them, including yours. It'll assign them in varying order, usually when you turn off your computer and another turns on. Meaning, they turn on, then request an IP address from the router in order to connect to the internet.
Your router sends any info that is requested out to the internet and when that info comes back, it needs a 'roadmap' to get that to your computer and not the other computers. So you need to know that if you have DHCP enabled, you're IP address might change at times. You'll either have to check from time to time and change the forwarding info, meaning retype in your new ip address when you get a different one or enable static ip's. It's up to you. Like I said, on one of those tabs, it'll give you a link to a manual in PDF form. That'll answer a lot of your questions. But don't hesitate to ask here.