Fischer was born in Hardwick, California, the daughter of Issei (Japanese immigrants) Chukuro and Kinko Tsubouchi. During World War II, as a young child, she and her family were forcibly evacuated from the West Coast and spent time in the Fresno Assembly Center before being relocated to Jerome and Rohwer internment camps. Fischer later lived in Chicago, Illinois, where as a young adult, she won the crown of "Miss Nisei Queen." She attended Rollins College from 1951-1953. She married Sy Fischer (an entertainment executive) on November 29, 1980 (her second marriage), and has three children. She resides in Los Angeles.
Fischer appeared in the stage production of The World of Suzie Wong in New York in 1958. She won a Drama-Logue Award for ensemble performance for Tea at the Old Globe Theatre in San Diego. She has also appeared in many productions with East West Players in Los Angeles, including Into The Woods. She toured the U.S. and Europe in The Peony Pavilion in 1997.
Fischer has appeared in many big-budget films, including Pirates of the Caribbean: At World's End (2007) as Mistress Ching, The Pursuit of Happyness (2006) as Mrs. Chu, Memoirs of a Geisha (2005) and War of the Worlds (2005). She has also worked on many independent Asian American films including Americanese (2009), Only the Brave (2005), Stand Up for Justice: The Ralph Lazo Story (2004) and Strawberry Fields (1997).