Computers store information and transfer it in binary, but display and accept from us in regular letters and decimal numbers.

Humans primarily use the decimal number system as it is related to how people tend to initially count and that is using our fingers. We have 10 fingers and there are 10 single digit numbers in the decimal number system. However not all humans have used decimal. The Babylonians used a 60 character system, why I do not know. However that is why hours have 60 minutes and there are 360 degrees in a circle.

Computes are electrical electronic devices and trying to store information in them directly using decimal numbers would be difficult as you would need to somehow represent the number as partially there or a electrical signal as partly on. Think of a light with an adjustable switch. It is hard to say the light is 25% on.

Binary is a number system that only has 2 single digit numbers. They are 1 and 0. That is easy to represent in electricity as either it is on (1) or it is off (0). Likewise when we store magnetically or burn in a CD/DVD, it is a positive or negative magnetic charge or there is or is not a hole burned in the CD/DVD.

The machine is storing and processing all the information in binary form, but we store it in binary from in the computer using a coding system called EBCDIC, ASCII or UNICIODE which converts the characters we know to binary. This happens automatically when you type in a number or letter. When we enter information in the computer it is converted to binary and then when it is displayed or printed for us, the computer does as it last task of converting from ASCII, EBCDIC or UNICODE back to the normal characters we know.

Normally we never think about how the computer actually carries the information as it does the conversions smoothly. If we had to do stuff on the compute in binary, very few of us would use a computer.