Network Intrusion Detection Systems (NIDS) are traditionally built to minimize the total number of misclassifications without considering financial implications. However, false positives and false negatives both impose monetary costs on an organization through wasted analyst time and damage from missed attacks. This work presents an approach which uses economically informed decision making to develop a cost-sensitive intrusion detection architecture that incorporates the cost of handling such misclassifications. Specifically, we propose a cost-sensitive supervised machine learning model alongside an economically informed thresholding technique to minimize the overall cost when dealing with cyber attacks. The models are evaluated across four unique scenarios in two environments, highlighting the broad suitability of the architecture. The various scenarios allow our architecture to be evaluated across a range of notoriously difficult to determine costs. Experimental results for the two domains demonstrate an average cost reduction of 59% over traditional accuracy-based intrusion detection systems. The trade-off, measured in reduced accuracy, is minor, with an average accuracy reduction of 1.25%. Our architecture allows organizations to make detailed and informed decisions about resource allocation when implementing security tools.