Google Duplex. Image synthesis technology (also, curiously, driven by AI). AI designed to scrape social media profiles. AI and bot accounts making and sharing relatable content.
Being able to create a believable human is so much closer than I think most of us realize, and this type of creation is important only to a very specific, technocratic authoritarian. Someone who envisions a world where every human being has functional purpose (choice being irrelevant, and this ideology being incompatible with liberty); a system designed to encourage compliance, blissful servility.
According to WaPo, “Duplex can only chat in ‘closed domains,’ which means it has a set purpose and a set script to achieve it.” What if that script isn’t commercial? What if its a flood of AI-driven calls to Discover Financial’s Customer Support, a social engineering attack designed to target peoples accounts by posing as them? What if the script is political, created by those who would sway us through viral marketing and moral contagion to think and believe what they want us to?
Google has John Legend’s voice. His fucking voice sticking to, and simulating a script. Sounding [almost] like a real person. The technology to impose his face on someone else’s body, and to put his voice that imitation, and to bring it to life in some heartfelt video about why you need to support XYZ---it is already here, now!
It may not be perfect, and many should still have the ability to discern a fake from the real thing, but for how much longer will that be the case while technology continues its march onward? In the wrong hands (in any hands) these media-related technologies have the power to do some real damage to the social fabric of society.
