Historical question. But this is what people do.
- Access the portal at https://portal.azure.com/#home.
- Find your "Network Security Group" and add your (client IP) IP for port 22.
- Make a note of the public IP - if it changes often-consider dynamic IP providers.
- You should have half of the keypair in a .pem file.
- Convert this to a .ppk file - See PuTTY support.
- Create a definition in PuTTY, go into Connection, SSH, Auth and enter the .ppk file details.
- Have a go. Refine fonts and character encoding later.
- Windows 10 has SSH from the command line, including SCP.
You should be good to go. Note that 2. is required every time if your client's public IP address changes. You can automate this with API scripts later. Don't just leave it open for everybody. I advise against password only authentication.
The Azure shell access from the web was not crash hot when this question was asked.