<p>Homeostasis occurs in a biological system when a chosen output variable remains approximately constant despite changes in an input variable. In this work we specifically focus on biological systems that may be represented as chemical reaction networks and consider their infinitesimal homeostasis, where the derivative of the input-output function is zero. The specific challenge of chemical reaction networks is that they often obey various conservation laws complicating the standard input-output analysis. We derive several results that allow us to verify the existence of infinitesimal homeostasis points both in the absence of conservation and under conservation laws where conserved quantities serve as input parameters. In particular, we introduce the notion of infinitesimal concentration robustness, where the output variable remains nearly constant despite fluctuations in the conserved quantities. We provide several examples of chemical networks that illustrate our results both in deterministic and stochastic settings.</p>