The simplest way to travel long distances is by coach, which should be booked a day or two in advance, especially if you wish to avoid a seat near the back which experiences a bumpier ride. Expect to pay about 6€ for a day's travel. If you start from Dhaka, there are a wide range of coaches to most corners of the country, while from other places, your options will be limited to a coach back to Dhaka and perhaps to 3 or 4 neighbouring towns. In Summer, "A/C" is most comfortable, which the rest of the year, "normal" (i.e. non-A/C) at half the price may be the preferred option. For shorter distances, local buses are a possibility, which you can hail from the roadside; though cheaper, they are much slower and so not recommended for long distance travel.
Trains are a possibility for some journeys. Though the routes are rather limited, the prices are reasonable, even if you pick one of the higher classes (recommended for foreigners). Boats are quite possible, and don't sink or capsize as often one might think from Western media reports... Hitchhiking is also possible, but is rarely practiced so may require some finesse.
Hotels are cheap, but of variable quality. You may prefer to randomly meeting people - Bangladeshis are extremely hospitable, and so if you talk to locals at all, frequent invitations to come and stay are to be expected.
Outside of Dhaka, expect food to be fresh, locally produced and spicier than in Europe or North America. Beware the local versions of fast food, most of which are not to be recommended.
Most people drink water from the ubiquitous tube wells. This water is generally good, although foreign stomachs may take a while to get used to it. Anything that comes through pipes or is stored in a tank should be boiled before being drunk. If you plan on staying in the country for days rather than weeks, bottled water is a pretty safe option. Note that red painted tube wells, are arsenious - and so only for washing, not drinking.
Foreigners will need a passport and quite likely also a set of passport photos to buy a SIM card. SIM cards are easily be set up to provide a passable (EDGE, 2.75G) service for most of the country, at a cost of about 10€/month. In the larger towns, internet cafes are available.
Bangladesh is an Asian country which doesn't have a very developed tourist sector. Visa rules are in a constant state of flux, but visas for extended stays are generally very difficult to obtain with local contacts.
Visa On Arrival
Getting a visa for Bangladesh on arrival (a.k.a. landing permit) at the International Airport in Dhaka may be possible, but this method is not recommended and should not be relied upon without clear investigation in advance. If available, they tend to be only for a short time (e.g. 14 days) and to be more expensive than a visa obtained in advance.
Single-entry tourist visas are generally available for 1-2 months stays. These are sometimes available in nearby countries, though this changes so up to date information is needed to prevent mishap. The simplest place to get a visa is in your home country (if an embassy is available). Many embassies accept postal applications, which may be easiest. Although not a well publicised possibility, once you are in Dhaka, it is often possible to extend tourist visas for a total of up to 3 months - and if you with to try this, the sooner you do this the better.
Passport holders of most countries find it difficult to stay in Bangladesh for more than 3 months with a Tourist (T) visa. Seek advice from local contacts about this possibility.
Other Visa Classes
Bangladeshi visas have different classes, issued by different branches of the Bangladesh government and governed by different regulations. Generally, this need not concern the tourist, whose options will be limited to a 'T' (tourist) visa. Users of other visas will probably not be responsible for their own visas anyway. For example, if you are traveling to Bangladesh for employment in the educational sector, an 'E' visa is applicable, in which case your employer should be able to sort this out for you.
Changing visa classes is problematic. For example, if you arrive with a 'T' visa, it is difficult to replace this with an 'E' visa. Generally, it is easier to leave the country and re-enter with an 'E' visa. Often this will entail a trip back to your home country, bearing requisite proof of employment or other supporting documentation for presentation at the embassy.
Overstaying your visa in Bangladesh can be an expensive business (a daily charge is applied), and may well prevent you from re-entering the country. To prevent problems, be aware of the possibility of disturbances such as hartals or random office closures and bear this in mind when scheduling travel.
Dhaka is the capital of Bangladesh, and the only place it is possible to extend your visa for that country. This is not a recommended procedure, but if you wish to try it, you should do so by proceeding to the Passport Building in Agargaon. Try to arrive in early morning, and be prepared to wait for a few hours queueing, and leave time to make more than one trip, as this may be required, as may passport photographs.