Start by contacting them with your account number and forwarding address. Most delays come down to them having no way to reach you.
For energy, suppliers must take steps to return credit on a closed account, and must refund promptly when you ask unless they have reasonable grounds not to.
If you changed supplier, Ofgem's guaranteed standards in Great Britain set deadlines on top of that: a final bill within six weeks, and your credit refunded within 10 working days of that bill. If your supplier misses them, compensation is due — currently £40, and a further £40 if that isn't paid within 10 working days. Those deadlines are built around a change of supplier, so if you've moved home rather than switched, ask your supplier what applies to your closure and to explain any delay.
Water and council tax work differently. Both should refund once the account is closed and they have your bank details — ask when that will be, and chase if the date passes.
If a company still won't pay, complain to them directly first, then:
Energy — the Energy Ombudsman, eight weeks after your complaint, or sooner if you get a deadlock letter.
Water in England and Wales — CCW, once the company has sent its final response or eight weeks have passed. CCW can help you resolve the complaint and explain how to access independent adjudication if it remains unresolved.
Council tax — in England, complaints about how the council has handled things can go to the Local Government and Social Care Ombudsman; disputes about who owes the council tax or how much normally go to the Valuation Tribunal instead. In Wales, the equivalents are the Public Services Ombudsman for Wales for complaints about how the council handled things, and the Valuation Tribunal for Wales for liability and banding. Check the appeal deadlines early — they're tighter than you'd expect.