The accused were nabbed by police after the latter approached them pretending to be buyers.
The accused Jossy hid six kilograms of gold in the form of four bangles concealed in 'Nivea Creme' boxes.
Seven people were arrested in the joint operations by forest intelligence, the Idukki forest flying squad and foresters of the Thodupuzha Range office on Thursday.
Three natives of Karnataka had brought the tusk to Mananthavadi to sell it in Kerala.
According to police, the accused were returning from Bengaluru purchasing MDMA and had taken the deviation from Gundelpett in Karnataka
During interrogation, the accused confessed that the drug was being smuggled from Bengaluru to Kannur for selling, said police.
At least three cases have been reported in as many weeks from the Kannur district.
The value of the seized saffron is estimated to be at least Rs 36 lakh.
The seizure was made on the basis of a tip received by the Directorate of Revenue Intelligence.
Among the four accused, two are absconding.