《big-data-Application.docx》由会员分享，可在线阅读，更多相关《big-data-Application.docx（5页珍藏版）》请在第壹文秘上搜索。
1、ApplicationofbigdataAbstract:Sincethemid1980s.theworldhasexperiencedanunprecedentedexplosioninthecapacitytoproduce.More,andcommunicaccdata,primarilyindigitalfmuts.Simultaneously,accesstocomputingtechnologiesintheformof(hepersonalPC.SnlarlPhone.andotherhandhelddeviceshasmirroredIhisgrowth.Wiihlhcscen
2、hancedCapabililiesofdatastorageandrapidcompulationaswellasreal-timedeliveryofinformationviatheinternet,theaveragedailyconsumpwercompaniesand(heircustomersconserveenergybygivingfeedbackmessagesFromamalfunctioningmetertothepowercompanyoralarmingcustomerswhentheyarcwastingenergy.CrimePredictionBigDataa
3、nalyticsishelpingPOliCCdepartmentstodeveloppredictionalgorithmstopreventprobableCrimCcommitments.TechnologyHadoopEcosystemVolume:largeamountofdataVariety:heterogeneityofdatatypes,representation,andsemanticinterpretation.Velocity:boththerateatwhichdataarriveandthetimeinwhichitmustbeactedupon.Veracity
4、:referstothebiases,noiseandabnormalityindata.Isthedatathatisbeingstored,andminedmeaningfultotheproblembeinganalyzed.(Inscopingoutyourbigdatastrategyyouneedtohaveyourteamandpartnersworktohelpkeepyourdatacleanandprocessestokeep,dirtydata,fromaccumulatinginyoursystems.)Challenges:Privacy:BiSDQIaprofoun
5、dlychangextheprMenofprivacyanotherreasonwhyIheUS,xdata-drivenCtvnpaniesareM)byinfsohardtousethereviewofdataprotectionlawtoweakenit.AbigdataSOCiCIyseemslbeinevilable.andPrVJniSCSmuch,butprivacy(pnperlyImderSkKMI)mustbeanimportantpartofanysuchsociety.First,weneedtthinkbroadlyaboutprivacyasmorethanjust
6、(hekeepingofasecret,butastherulesthatmustgovernPerSOnalinformation.Second,weneedUrealize(hatinformationdoesnlloselegalPrUkXtionjuslbecauseitisheldbyanotherperson.Confidentialityruleswillbeessentialinanyb设datafuture.Ttiini.wcneedtodevelopsomebigdataethicsasasociety.BigdatainthefieldofEducation:Studei
7、nsusingonlinesystemSystemtocollectandrecordtheStUdCnlSonlinelearningbehavior.savinginlodatabaseFordataanalysisandprocessing,predictingthestudents,academicperformanceTheresultoftbcforecastandfeedbackforvisualprocessingToniecetherequirvmentsofthestudents*individuallearningmaterialsTeacher.dminisirator
8、s.anddeveloperstimelytogivestudenlsBUkklnCCandhelpBigdatahaschangedthewayfinancialinstitutionsoperateintraditionaldatafromfourdifferentways，achievinggreatbusinessvalue。Thesefourareas(4C)include：CompatibilityConnectedness*Cost*Capitalization.Theapplicationoflargedatainfinancialindustryisgraduallyexpa
9、nding,Overseas,bigdatahavebeenfullytriedinthefinancialindustry,suchasriskcontrol,operationmanagementalessupportandbusinessmodelinnovation.InChina,theapplicationoflargedatainfinancialinstitutionsisstillbasicallyattheinitialstage.Keychallengessuchasdataconsolidationanddepartmentalcoordinationremainthe
10、mainbottlenecksthathinderfinancialinstitutionsfromtranslatingdataintovalue.Example:AnonlinestorehasanexistingSQ1.Databasethatmanagesitsauthorizationsystemandstorcf11Hit.SQ1.isnecessaryinordertoensurenolosttransactionsandfullACIDcompliance.Thestorealso(racksitssocialaclivilyusingaKEY/Valuedalabasc.Th
11、isdalabasePrwiIkSspeedandcaneasilystorelargeamountsofdatabutPrOVidCSlessconsistencythanaSQ1.SOIUtion.However,associaldataisnoascriticalastransactional,thissolutionisappropriate.Thestorealsoprovidesrvconuncndalionstocustomersaboutproductstheymightlike.TheyuseaColumndatabaseforthispurpose.SQ1.isnotncc
12、essar)becausetheinformationisnotrelational.TheMinistryofEducationisresponsibleforInanagingdataaboutstudentsaround(heprovince.Theyhaveover125millionrecords(students).241millionrowsandover15yearsworthofdata.hcdataisstoredusing450tables,buttheonlyvarietyisatattributelcvcl(i.c.StandardizedtcMing,pcrtbnn
13、ancc.ccc).Theyreceivethedatafromindividualschoolsingiantchunksfourlimesayear.TheMinistrydeckledagainstusingbigdatatoolstomanagetheirdataandisstillusingaRDBMS.Ahhoughtheyhavalareamoun(sofdata,the(yeoftheirdataandIbevelocityatwhichIheyreceiveitmeansthataRDBMSisstillbest.TheyalsohavecomplianceandprivacyissuesthatmakeNoSQ1.anunsuitablechoice.EachschoolandschoolboardwassendingthedataoIheMinistryusingdifferentschemasand(hisinconsistencymadedatamanagementdifikuhanddecreasedthevalueofIhedataforBipurposes.TheMinistrylookamasterdatamanagementapproach.Overthecourseof4years,theycreatedcommonmclhgiesa
