After 9/11, the country was drawn into a familiar debate. With a war on terrorism ramping up, we had to ask how much authority we would give the government to protect us. How much personal information would we be willing to share? How much safer would a security measure have to make us to justify diminishing our privacy? I’m not sure we fully answered those questions, or that they’re fully answerable.

But Person of Interest gave us another chance to think about them. The idea is that a reclusive billionaire (Harold Finch) has developed a machine that can predict crimes in New York City before they happen. He and his partner (John Reese) work together to save the person in danger. To make its predictions, the machine must rely on surveillance footage from around the city. In other words, someone is always listening to your calls or reading your emails.

In the first two seasons, this seems like a good thing. Finch and Reese stop several murders before they happen. But what if the machine is not in good hands? In season 3, another artificial intelligence named Samaritan comes online at the behest of John Greer, someone who thinks he has humanity’s best interests at heart. He survived the German destruction of London during World War II and is convinced that humanity cannot peaceably govern itself. The problem is that Samaritan’s willing to do anything. It’ll hack elections–in one episode, it rigged vote totals in New York so that its preferred candidate could win the governorship. And it’s willing to kill.

Would we rather live in a world with such an artificial intelligence? It can stop all crime and stop wars if we allow it to monitor us nonstop and run our lives for us. That is, if we give up our liberty, it can give us security. This might seem more palatable when it’s Finch’s machine because Finch is a good man. He programmed it with the best of intentions. But it is easy to see any number of governmental officials justifying their encroachments on our freedom and privacy that way. They would tell themselves that the tools they create are meant to keep us safe, and that they would never abuse them. And they probably mean it a lot of the time.

But once a technology is created, bad people as well as good can use it. Once the machine was created, someone would inevitably use the knowledge gained to build Samaritan. As we continue to contemplate how to deploy new tools to protect us from crime and terrorism, Person of Interest reminds us to ask “what if someone who isn’t so noble gets their hands on this?”