Since the advent of the 5.56x45 mm. NATO cartridge, the world has essentially gone into three basic calibers as far as militaries are concerned: the NATO 5.56x45 mm. (used by M16, Steyr AUG, FAMAS, L85, INSAS etc.), the Russian 5.45x39 mm. (used by AK74 family) and the Russian 7.62x39 mm. (used by AK-47, AKM and clones such as the Type 56). The first two of these are generally used by most military forces, whereas the third cartridge is generally widespread among many insurgent groups due to the popularity and wide-spread nature of the AK-47 and AKM family of assault rifles. However, there has been some other significant research in cartridge developments as well. Both the UK and the US did some work to find an intermediate cartridge between the NATO 5.56x45 mm. and the older NATO 7.62x51 mm., in order to strike a balance between bullet effectiveness and recoil force. This was due to the 7.62x51 mm. cartridge having too much power and weight and the 5.56x45 mm. not having sufficient range for some applications. Experiments showed that cartridges such as the 6x45 mm., the Grendel 6.5x38 mm. or the Remington SPC 6.8x43 mm. strike a pretty good balance by having more range than a 5.56x45 mm. cartridge, but still having relatively less recoil and less weight than a 7.62x51 mm. cartridge. With combat in Afghanistan taking place over longer ranges and the advent of scopes on assault rifles, it is possible for an infantryman to engage over longer ranges now and hence there have been noises made in various quarters to replace the M16 with a newer rifle using one of these cartridges instead.
Newer bullet designs using exotic technologies were also tried out during the 1970s and 80s, but they've been much less successful in this regard. For instance, there were attempts to design cartridges that fired flechette darts instead of conventional bullets. However, the cost of ammunition was prohibitive and thus this never became popular. Another exotic concept was folded ammunition, which was ammunition that was roughly U-shaped. The idea was to reduce the length of the cartridge in order to speed up the firing cycle of rifles.
Folded ammunition examples. Click on images to enlarge.
Folding ammunition never caught on either. Another concept was the caseless ammunition cartridge, for which a lot of work was done by Hecker & Koch and Dynamit industries. The idea is that since a caseless cartridge doesn't have a brass case, there is no need for the rifle to eject it after every shot, which reduces the number of steps in a firing cycle and thereby enables faster firing rates. H&K developed the G11 assault rifle to use caseless ammunition. The concept of a consumable cartridge is actually a very old one, as it was used by the Dreyse Needle Gun of 1835! However, with automatic weapons and modern ammunition, there are more problems to solve. For one, caseless cartridges are more easily damaged by rough handling since they don't have a hard outer case and two, the brass case removes some of the heat from the chamber of the firearm and the lack of a brass case means that caseless ammunition could cook-off in a hot chamber. H&K solved the first problem by putting cartridges in a sealed plastic case and had to spend a lot of time developing special propellants to solve the second issue. The G11 was about to be adopted by the West German military when the Berlin wall came down and the cold war ended. This resulted in cutbacks in military spending and H&K went into financial difficulties as a result and was acquired by the British, where they earned their keep by helping fix problems with the British L85 assault rifles. Interest in caseless ammunition technology has been renewed since 2004, due to the US military's Lightweight Small Arms Technology (LSAT) program.
Oddly enough, despite most of the new rifles being bullpup layouts, the US military is looking at conventional layouts for their M16 replacements. For instance, the US Army has switched to the M4 (which is the carbine form of the M16). The (now cancelled) XM8 project, which was designed to replace the M16, also had a conventional layout:
XM8 Assault Rifle. Click on image to enlarge. Public domain image.
The FN SCAR, adopted by the US Special Operations Command (SOCOM) as well as the US Marines M27 Infantry Automatic Rifle also use conventional layouts. Thus, it appears that the US is definitely not bucking the bullpup-layout trend, at least in the near future.
Another interesting concept developed in Australia is the Metal Storm rifle, where many bullets are stacked head to tail in series inside a barrel, with propellant between the bullets. Ignition of individual cartridges is accomplished electronically. Since the cartridges are completely consumable and because they are stacked one behind the other in line, there is no need for case ejection or a feed system to load new cartridges in the chamber. This makes the firing rate much faster than other designs, as well as contributing to reduced weight. As of now, Metal Storm products have found limited use and support from the US Marines.