Pourrais-je avoir un peu de lait? Why is it not 'du lait' as in "pourrais-je avoir du lait?"

Because there is an amount specified for the milk ("un peu") you don't use the article, just "de". If you say "je voudrais du lait' you're just asking for some milk, no amount specified, hence the article which makes "de" to "du". There is a specific lesson dedicated to the partitive article. -- Chris (not a native speaker).

Thank you, I now understand.

Bonjour Stewart !

Indeed, with expression of quantity like "beaucoup de, un peu de...", you will only use "du" if you mean "a lot of the, a bit of the", as such:

Here's our lesson about partitive articles:
https://french.kwiziq.com/revision/grammar/du-de-la-de-l-are-used-to-express-some-or-any-partitive-articles

I hope that's helpful!
À bientôt !