• 8 Posts
  • 887 Comments
Joined 1 year ago
cake
Cake day: June 4th, 2023

help-circle









  • Volume is only useful for things that are non-compressible (i.e. fluid). What if you’re measuring flour? Usually, the measurements are given for sifted flour, but that’s not something you would know unless you’re experienced in the kitchen. And even if you do sift your flour, there’s still going to be a lot of variation depending on how much things get compressed again as you’re scooping it out.











  • I’m not familiar with the term “beam” in the context of LLMs, so that’s not factored into my argument in any way. LLMs generate text based on the history of tokens generated thus far, not just the last token. That is by definition non-Markovian. You can argue that an augmented state space would make it Markovian, but you can say that about any stochastic process. Once you start doing that, both become mathematically equivalent. Thinking about this a bit more, I don’t think it really makes sense to talk about a process being Markovian or not without a wider context, so I’ll let this one go.

    nitpick that makes communication worse

    How many readers do you think know what “Markov” means? How many would know what “stochastic” or “random” means? I’m willing to bet that the former is a strict subset of the latter.